Patrick James

As a biotech/pharma quality professional (16 years: cleanroom operations, deviation/CAPA investigations, training, audit support), I address Questions 1, 2, 9, 10, 14, 16, 18, 19, 21, 22, 24, 25, and 26. Key points: add failure observability to the risk framework; require independent, held-out benchmark data with prespecified criteria; define human oversight as explicit attributable approval; monitor the foundation model, not just the device wrapper; capture near misses; use independent third-party sample review; treat prompts as validated configuration under PCCPs; and support foundation-model master files with change-notification commitments. Full comment attached.

Public Comment — FDA Docket FDA2026-N-7874
Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper
and Request for Feedback

Submitted by: James Patrick Submitted as an individual commenter.

Comment addresses Questions 1, 2, 9, 10, 14, 16, 18, 19, 21, 22, 24, 25, and 26.

I am a quality professional with sixteen years in biotech and pharmaceutical manufacturing,
including cleanroom operations, deviation and CAPA investigations, training, and audit support. I
address the questions below, which match my expertise. My comments come from the practical
side of regulated AI: what it takes to validate a system, keep its records trustworthy, and catch
failures before patients are harmed. I support the direction of this discussion paper and offer
the following.

On risk assessment (Q1, Q2)
Q1 — a risk dimension not captured: observability of failure. The two-axis framework (device
activity by consequence of relying on an incorrect output) is a sound starting point. I suggest the
framework also make explicit whether failures are observable and attributable. A low-severity,
high-autonomy function can quietly generate incorrect records that feed downstream decisions.
In manufacturing quality, we treat this as a data-integrity problem, not just a clinical one: a
wrong value nobody sees is worse than a wrong value a clinician questions. Risk placement
should account for whether outputs are routinely examined by a competent human, whether
failures are detectable, and whether the output record is complete and attributable (Q2: the risk
modifiers for informational functions — wording, specificity, personalization, context — are a
natural place to capture this observability evidence, and uncertainty in it should be stated, not
smoothed over).

On device benchmarking (Q9, Q10)
Q10 — benchmark independence and contamination. The paper rightly worries about data
contamination and about sponsor-developed benchmarks being optimized to the test. The fix is
familiar from validation practice: benchmark data must be held out from development and
controlled like any other qualification data, with acceptance criteria prespecified before the test
is run, the same way a validation protocol prespecifies its criteria before execution. If the test
set is visible to the developer, the benchmark measures memorization, not competence. (Q9:
the Appendix A elements are a good list; I would add adversarial and edge-case coverage as an
explicit element — deliberately difficult cases, including ambiguous inputs and inputs at the
boundary of the intended population, and a defined method for grading "wrong but plausible"
outputs, since plausible error is the characteristic failure mode of generative systems.)

On human oversight and performance standards (Q14)
Q14 — define human oversight as explicit, attributable approval. For device functions that
generate clinical recommendations or take actions, oversight should mean a qualified person
explicitly approves consequential outputs, with the record showing what was actually reviewed,
when, and with what authority to override. From Part 11 practice: a single click that approves a
thousand outputs is not a signature on any of them. Where a system operates with meaningful
autonomy, the system must be able to stop when the human cannot intervene in time. When
the comparator is human-AI team performance rather than the device alone, the evaluation
should specify what the human in that team actually reviewed — team performance built on
rubber-stamping is device-alone performance wearing a costume.

On postmarket monitoring (Q16, Q18, Q19, Q21, Q24)
Q24 — monitor the foundation model, not only the device wrapper. When the device relies on
a third-party foundation model, that model's updates, behavior shifts, and failure patterns are
the device's postmarket reality. A monitoring plan that watches only the wrapper will miss drift
that originates upstream. (Q18: degradation thresholds should be prespecified, and the plan
should define who is notified when they are crossed.) Two more signals worth requiring: near
misses — generative outputs caught before causing harm are the most informative safety signal
this technology produces, and adverse-event reporting alone will undercount failures of systems
whose errors are frequently intercepted by vigilant clinicians; and independent sample review
(Q16, Q19) — periodic sample-based clinician review should be genuinely independent, with
audit-ready review records. From investigation experience: the organization that owns a system
has inherent incentives to soften the recorded root cause when something goes wrong. For
high-autonomy functions, samples and reviewer qualifications should be available to the agency
on request. Responsibility for monitoring should be allocated across the chain (Q21), but the
device manufacturer cannot delegate away accountability for a component it chose.

On PCCPs and change control (Q22)
Q22 — a prompt is part of the validated configuration. Prompt-level modifications, retrieval
changes, and foundation-model updates can each change device behavior as materially as a
code change. PCCPs should treat them as modifications and define which classes of change
trigger re-benchmarking against the locked acceptance criteria. Cumulative and interactive
effects are the real hazard — no single change looks significant, but the sequence drifts the
device. PCCPs should define a re-benchmarking trigger based on the number or combined scope
of changes, not only on each change in isolation. Where the foundation model is controlled by a
third party, the device version should reference a locked model version and locked
configuration, with any update treated as a controlled change.

On third-party foundation models (Q24, Q25)
Q25 — support the device Master File concept. A foundation-model MAF would help, provided
it contains what a device manufacturer actually needs to validate against: locked model identity
and version, training-data description sufficient to assess representativeness, known failure
modes and limitations, and a change-notification commitment. (Q24: pharmaceutical
manufacturing solved the analogous problem with supplier qualification — you cannot
outsource the validation of a component you depend on. Agreements with model providers
must support timely notice of changes and access to safety-relevant information. If a provider
cannot supply that transparency, its model should not underlie a device in a higher-risk region of
the framework.)

On agentic AI systems (Q26)
For systems that plan and execute multi-step tasks, use external tools, or take actions in
sequence, each step is a validation point and each handoff between steps is a failure point. The
evaluation should bound the action envelope: the defined set of tools, the permitted action
types, and the stopping conditions. Chains of related actions must be logged per step with
attributable records, so that when something goes wrong the failure can be reconstructed
rather than inferred. Autonomy should scale inversely with the severity of what the system is
permitted to do without a human decision in the loop.

Submitter note
Submitted as an individual commenter. I am an independent quality professional; this comment
reflects my professional views only and was prepared without access to any proprietary
information of any former employer.