Benjamin Speck

See attached file.

FDA Feedback Preparation Document
Opening Statement
To the U.S. Food and Drug Administration (FDA)

Re: Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion
Paper and Request for Feedback. Docket Number: [To be inserted by submitter if
applicable].

My name is Benjamin Speck, and my professional experience includes the review of medical
device technical documentation, clinical evaluation, risk management, software as a medical
device (SaMD), AI-enabled medical devices, and postmarket surveillance. My work has also
involved assessing compliance with European regulatory frameworks, including Regulation
(EU) 2017/745 (MDR), the UK Medical Devices Regulations, and the legacy AIMDD and
MDD. I appreciate the opportunity to provide feedback on this discussion paper and
commend FDA for engaging stakeholders as it develops a proportionate, risk-based
regulatory approach for generative AI-enabled medical devices.

FDA source checked: Considerations for the Regulation of Generative AI-Enabled Medical Devices, Discussion Paper
and Request for Feedback (2026), https://www.fda.gov/media/194242/download.

Discussion Questions
1. Does the two-axis risk framework, organized around device activity and the consequence
of relying on an incorrect output, appropriately capture the dimensions most relevant to the
risk of a GenAI-enabled software function?

If there are additional dimensions—such as

• the reversibility of a resulting action,
• the availability of downstream safeguards,
• the time pressure of the deployment setting, or
• the traceability of the output (i.e., to primary source materials)—

that should be represented in a risk framework, please describe and provide examples of
how they should be represented.

The proposed two-axis framework resembles the approach used in IMDRF and
MDCG 2019-11, which considers both the significance of the information to the
healthcare decision and the severity of the patient’s condition. The comparison
below illustrates the general alignment.
 Risk Level / FDA Considerations for MDCG 2019-11 IMDRF Definition
State of the Regulation of
Health Generative AI-Enabled
Medical Devices
Low (Limited) Informational: Non- Informs Clinical Information will not trigger an immediate
Directive Management or near term action...
• To inform of options for
treating, diagnosing, preventing,
or mitigating a disease or
condition
• To provide clinical information
by aggregating relevant
information
Medium Informational: Action- Drives Clinical Information will be used to aid in
(Moderate) Directing Management treatment, diagnosis, triage, or identify
early signs of a disease or condition which
will be used to guide next diagnostics or
next treatment interventions...
• To aid in treatment by
providing enhanced support to
safe and effective use of
medicinal products or a medical
device
• To aid in diagnosis by analyzing
relevant information to help
predict risk of a diseases or
condition or as an aid to making
a definitive diagnosis
• To triage or identify early signs
of a disease or conditions
High (Severe) Action-Taking: HCP Treat or Diagnose Information provided will be used to take
Supervised an immediate or near term action...
• to treat/prevent or mitigate by
Action-Taking: Fully connecting to other medical
Autonomous devices, medicinal products,
general purpose actuators or
other means by providing
therapy to a human body
• to diagnose/screen/detect a
disease or condition
IMDRF/SaMD WG/N12FINAL:2014

The framework is a useful high-level guide and is broadly consistent with the risk
concepts used in IMDRF and MDCG 2019-11. FDA should make clear, however, how
the framework fits with existing device classification, ISO 14971 risk management,
IEC 62304 software lifecycle controls and IEC 62366-1 usability engineering. It
should guide the level of evidence and oversight without creating a separate or
conflicting risk system.

The framework should give greater weight to the intended user, use environment,
position in the clinical pathway and urgency of the decision. These factors often
determine whether software merely informs care, directs the next clinical action or
effectively drives diagnosis or treatment. A recommendation used by a specialist
during routine review presents a different risk from the same recommendation
delivered to a patient or generalist during an urgent situation.

FDA should also distinguish characteristics of the intended use from risk controls.
Reversibility and downstream safeguards may reduce the consequences of a failure,
but only when those safeguards are available, effective and validated for the
 intended users and environment. Traceability to source material is an important
control, but it should not be assumed that a user will independently verify every
output from a cleared or approved device.

Additional modifiers may include whether the model changes after deployment,
whether one output affects a single patient or an entire network, and whether the
clinical workflow provides independent information that can confirm or contradict
the output. These factors should be used to refine the two-axis framework and to
justify proportionate verification, validation and postmarket monitoring.

2. CDRH seeks input on how to account for the spectrum of GenAI-enabled informational
functions that vary in the degree to which they direct a user to a particular action (i.e.,
between “non-directive” and “action-directing”).

What characteristics of an output—such as its

• wording,
• specificity,
• personalization, or
• context—

could be considered as modifiers of the risk associated with an informational function, after
accounting for the device’s overall functionality and intended use?

What additional information would provide manufacturers with sufficient clarity and
predictability around risk assessment for informational functions while recognizing that
directiveness may exist along a continuum rather than as a binary distinction?

Directiveness should be treated as a continuum rather than a binary category. The
risk depends not only on whether the information is correct, but also on how
strongly the wording, presentation and clinical context encourage the user to take a
particular action. A confident, personalized recommendation presented at a timecritical point in care may be more influential than a general statement of possible
options.

Relevant characteristics include imperative or definitive language, specificity,
personalization, expressions of uncertainty, confidence scores, supporting rationale,
presentation of alternatives, and the location of the output within the clinical
workflow. The intended user also matters because a patient, nurse, general
practitioner and specialist may interpret and act on the same wording differently.
User trust and reliance may also increase over time.

Manufacturers should define these characteristics in the use specification and
evaluate them through the IEC 62366-1 usability-engineering process. The riskmanagement file should address foreseeable misunderstanding, automation bias
 and overreliance, while validation should confirm that representative users
interpret and act on outputs as intended under realistic conditions.

FDA should provide examples showing how these factors affect the level of
directiveness and the expected evidence. Guidance should also explain when
variable or non-binary outputs require larger or more diverse summativeevaluation samples based on the intended users, use environment and clinical
consequences.

3. CDRH seeks input on whether and when a GenAI-enabled function that results in delivery
of clinical information to patients, as opposed to HCPs, could present different or higher
risks, while also recognizing the potential benefits associated with improved patient
empowerment, engagement, and access to clinical information.

What device characteristics, output features, or safeguards might mitigate risks that could
arise when a user lacks the domain knowledge to independently evaluate an output,
without unnecessarily underestimating patient capability?

Patient-facing GenAI functions may present different or higher risks than
comparable HCP-facing functions because a healthcare professional may not be
present to interpret the output, recognize an error or place the information in
clinical context. This does not mean that patients are incapable of using clinical
information. It means that the device must be designed and validated for the
knowledge, health literacy, language needs and circumstances of the intended
patient population.

The level of risk should depend on the intended use, disease severity, urgency and
consequences of foreseeable reliance. Manufacturers should evaluate whether an
incorrect, incomplete or overly reassuring response could cause delayed care,
inappropriate self-treatment, unnecessary healthcare use or failure to respond to
worsening symptoms. Patient reliance on the output should be treated as a
foreseeable use condition, not as misuse that can be addressed only through a
disclaimer.

Risk controls should become stronger as potential harm increases. Controls may
include plain-language limitations, clear expressions of uncertainty, symptom-based
red flags, instructions for when and where to seek care, confirmation that the
patient understood critical information, and direct escalation to a healthcare
professional or emergency service when appropriate. These controls should be
validated through usability testing with representative patients and monitored
through postmarket surveillance.

4. CDRH seeks input on whether and how the distinction between generalist and specialist
physicians might inform the assessment of risk for HCP-facing GenAI-enabled functions.
Under what circumstances, if any, might risk be affected when an HCP who lacks the
relevant clinical specialist knowledge receives information that falls within a specialist area
of practice?

What device characteristics or safeguards might mitigate such risks?

The distinction between a generalist and a specialist may affect risk, but the more
important question is whether the intended user has the knowledge and experience
needed to understand, question and safely act on the device output. A function
designed for a specialist may present greater risk when used by a generalist who
cannot recognize an incorrect recommendation, an important omission or a
limitation of the underlying evidence.

This issue is not unique to GenAI. Manufacturers should clearly define the intended
users and clinical scope, then demonstrate through usability engineering, clinical
evaluation and performance validation that those users can safely interpret the
output under expected conditions of use. Validation should include representative
users with the least experience permitted by the labeling, not only highly
experienced clinicians.

Safeguards may include limiting the scope or specificity of recommendations, clearly
communicating uncertainty, presenting supporting evidence, requiring specialist
review for defined scenarios, and providing user-specific labeling or training. The
appropriate controls should depend on the intended use, clinical environment,
urgency and severity of harm if the output is misunderstood or relied upon
incorrectly.

5. How should risk be assessed for multi-turn conversational systems whose behavior
emerges across a conversation?

Risk assessment for multi-turn conversational systems should evaluate the
conversation as a whole. A response that appears acceptable by itself may become
unsafe when earlier information is forgotten, new symptoms are not incorporated,
recommendations change without new evidence, or repeated questioning weakens
an appropriate safety message.

Manufacturers should identify foreseeable conversational pathways within the ISO
14971 risk-management process. These should include incomplete or contradictory
information, extended interactions, user attempts to move the system beyond its
intended use, repeated challenges to escalation advice, prompt manipulation and
increasing user reliance. The assessment should consider how these behaviors
could lead to false reassurance, delayed care, inappropriate escalation or
unsupported clinical decisions.

For example, a patient may initially report mild symptoms and receive routine
advice, then disclose a serious symptom later in the conversation. The system
 should reassess the complete clinical picture and escalate appropriately. Similarly, if
a user repeatedly challenges an urgent recommendation, the system should not
gradually soften that recommendation unless new clinical information supports the
change.

Verification and validation should test complete conversations, not only isolated
responses. Testing should include long-duration and simulated clinical interactions,
contradictory or incomplete information, adversarial prompts, attempts to bypass
intended-use limitations, and repeated challenges to safety recommendations. The
manufacturer should confirm that the system retains critical facts, updates its
assessment when new information is introduced, maintains appropriate escalation
advice and stays within its validated clinical role.

Acceptance criteria should address clinically meaningful conversational failures,
such as loss of critical context, unsupported changes in certainty, contradictory
recommendations and conversations that end outside the intended use. Testing
should use the complete device configuration, including the model, prompts,
retrieval sources, tools, guardrails and user interface. Postmarket monitoring should
also evaluate conversation-level complaints, drift and emerging patterns of user
reliance.

6. How should under-escalation and over-escalation be characterized and weighed for care
escalation functions?

Under-escalation and over-escalation should be evaluated as separate hazardous
situations, not simply as opposite classification errors. For each scenario, the
manufacturer should consider the device’s intended use, use environment, intended
users, point in the clinical-care pathway and the urgency of the next clinical step.
These factors determine both the type and severity of harm. For example, overescalation that leads only to additional laboratory testing presents a different risk
from over-escalation that leads to an unnecessary surgical intervention. Similarly,
under-escalation may range from a short delay in routine follow-up to a missed
emergency referral that causes irreversible harm.

The point at which the technology is used in the clinical pathway should therefore
be a central part of the device-specific risk assessment. Manufacturers should map
how an output affects subsequent decisions, including who receives it, how quickly
action is expected, whether a qualified professional reviews it, and whether the
resulting action can be reversed. Separate clinical endpoints, severity ratings,
acceptance limits and statistical uncertainty should be defined for under-escalation
and over-escalation. For example, a chest-pain triage function should separately
measure missed urgent referrals and unnecessary emergency referrals, with
independent clinical adjudication of atypical presentations.
 This analysis could be incorporated into an enhanced EN 62366 usabilityengineering workflow that maps the device’s use within the clinical pathway,
identifies escalation-related use scenarios and evaluates the consequences of
foreseeable user actions or reliance on the output. FDA should consider publishing
guidance on this approach or supporting an appendix to EN 62366 that addresses
AI-enabled escalation functions. Manufacturers should also use red-flag rules, tested
escalation language and clear human handoffs, continuously monitor both error
directions, define investigation thresholds, and establish corrective and preventive
action or suspension criteria.

7. Is the competency-based approach (benchmarking plus clinical confirmation)
appropriate?

A competency-based approach can be useful, but it should not be treated as proof
that a device is safe for clinical use. A strong benchmark score only shows how the
device performed on a defined set of test cases. It does not show how the device will
perform with actual patients, intended users, time pressures, incomplete
information or real clinical workflows.

Manufacturers should first define exactly what the device is expected to do, what it
must not do, who will use it, where it will be used and which errors could cause
harm. The evidence should then combine three elements: independent benchmark
testing using a locked test set; clinical confirmation using representative real-world
data and realistic conditions of use; and ongoing monitoring after deployment. The
amount and strength of evidence should be proportionate to the device’s risk,
autonomy and role in patient care.

For example, a device that drafts radiology reports should not be evaluated only on
grammar or how closely its wording matches a reference report. Testing should
determine whether it misses clinically important findings, adds false findings,
communicates uncertainty appropriately and supports safe review by the
radiologist. The manufacturer should document the complete version tested,
including the model, prompts, retrieval sources, guardrails and user interface,
because changes to any of these components may affect performance.

This approach should supplement—not replace—the manufacturer’s ISO 14971 risk
management, IEC 62304 software lifecycle controls, IEC 62366-1 usability
engineering and device-specific clinical evidence. Regardless of whether testing is
performed internally or by an independent third party, the manufacturer remains
accountable for the safety, performance and regulatory compliance of the complete
device.

8. How should the two-axis risk framework influence evidence expectations?
 The two-axis framework should serve as a general guide for scaling evidence
expectations as device risk increases. As a device moves from providing nondirective information toward directing clinical action or acting autonomously—and
as the consequences of an incorrect output become more serious—the amount,
strength and independence of the supporting evidence should increase.

For lower-risk, non-directive functions, targeted analytical testing, usability
evaluation and confirmation of basic safety controls may be sufficient. Devices that
direct clinical decisions should require stronger evidence showing how the device
and intended users perform together in realistic clinical conditions. High-risk or
autonomous devices should require the most rigorous evidence, including
representative clinical validation, end-to-end system testing, independent review,
fail-safe testing and robust postmarket monitoring.

The framework is not intended to prescribe one fixed study design or replace
device-specific risk management. Instead, it provides a clear principle: evidence
requirements should escalate with the device’s clinical influence, autonomy and
potential consequences of failure. Manufacturers should use the framework to
justify why their testing and validation are proportionate to the device’s risk and
sufficient to support its intended use.

9. Does the proposed benchmarking structure provide adequate evidence of safety and
effectiveness?

Benchmarking is a useful tool for demonstrating specific aspects of device
performance, but it does not by itself establish safety and effectiveness. The
evidence needed depends on the device’s intended use, clinical application, level of
autonomy and the harm that could result from an incorrect output. Benchmarks can
test defined capabilities and failure modes—such as hallucinations, omissions,
inconsistent answers, unsupported citations and refusals—but they generally do not
evaluate how an intended user interacts with the device or how the device performs
within an actual clinical environment (i.e., clinical evidence).

Summative usability evaluation can simulate the intended clinical workflow and use
environment, but even a well-designed simulation may not fully reproduce the time
pressure, workload, interruptions, competing priorities and emotional stress
experienced in real patient care. These factors can affect how users interpret, verify
and act on AI-generated information. There is therefore no true substitute for
appropriate clinical evidence when the device’s safety and effectiveness depend on
performance under real clinical conditions.

FDA should apply a risk-based approach when determining the appropriate balance
between benchmarking, simulated-use evaluation and clinical evidence. Lower-risk
devices may be supported by strong benchmarking and usability evidence when the
consequences of failure are limited and effective safeguards are available. Higher8
 risk devices—and AI functions that are more autonomous, more influential in
clinical decisions or capable of causing serious harm—should be held to a higher
evidentiary standard and require stronger clinical confirmation. Benchmark results
should remain clinically relevant, use representative inputs and independent
reference standards, include acceptance criteria tied to potential harm, and evaluate
important subgroups. Locked validation data should also be separated from
development data and protected against contamination. Clinical evidence and
postmarket monitoring should address the safety and performance questions that
benchmarking cannot answer.

10. How should benchmark validity and real-world representativeness be established?

Manufacturers should clearly explain where the benchmark data came from, which
cases were included, when and where the data were collected, and whether any of
the data may have been used to train the model. The benchmark should attempt to
represent the patients, users, clinical settings and disease frequency expected in
actual use. This population should be defined and justified in a manner similar to the
population selected for a clinical evaluation or the intended-user population used in
an EN 62366 summative usability evaluation. Manufacturers should explain why the
selected participants, clinical characteristics, user groups and use environments
adequately represent the device’s intended population and conditions of use. The
benchmark should also include difficult but realistic cases, such as rare serious
conditions, incomplete or conflicting information, different languages and cases that
differ from the model’s training data.

Qualified independent clinical reviewers should confirm the correct answers, and
manufacturers should use controlled, versioned datasets and a consistent testing
process. A benchmark based mainly on large specialty hospitals, for example, may
not show how the device will perform in rural clinics, community settings or when
used directly by patients. After deployment, manufacturers should compare realworld users and cases with the population used for validation. Meaningful
differences, performance decline or evidence of drift should trigger investigation
and, when necessary, corrective action.

11. How should sponsors select and justify clinical confirmation approaches?

Sponsors should choose the type of clinical study based on how the device will be
used and the seriousness of the harm that could result if it is wrong. Depending on
the risk, the study may use existing patient records, simulated clinical scenarios,
silent-mode testing in which the device does not influence care, a prospective study
or a controlled clinical study. The sponsor should explain how the chosen approach
will identify clinically important errors, overreliance by users and problems in the
clinical workflow.
 Before the study begins, the sponsor should define the main outcomes, comparator,
sample size, relevant patient subgroups, review process and rules for stopping the
study. The amount of evidence should increase with the device’s risk and level of
autonomy. For example, a tool that drafts discharge instructions may primarily
require testing for patient understanding and missing or incorrect information. A
device that independently recommends treatment would require much stronger
evidence of clinical performance, patient safety and performance within the
complete healthcare system. Human-factors testing should also reflect the intended
users, clinical setting, time pressure and actual conditions of use.

12. How should statistically meaningful performance measurement be achieved?

Manufacturers should define in advance exactly what will be measured, how success
will be determined and what level of performance is clinically acceptable. The
analysis should account for repeated tests involving the same patient, related
prompts, missing data and differences of opinion among clinical reviewers. Overall
accuracy should not be used by itself because a high average score can hide rare but
serious failures. For example, the device’s ability to recognize and escalate an
emergency should be reported separately from its performance in routine-care
situations. Results should also be reported for relevant patient groups to identify
meaningful differences in performance or bias. Because a generative AI system may
produce different answers to the same input, testing should include repeated trials
using the same fixed device configuration.

Sample sizes should be based on how often important events are expected to occur
and the level of statistical confidence needed, rather than an arbitrary number of
prompts. After deployment, monitoring methods and alert thresholds should also be
tested to ensure they can identify real changes in performance without generating
excessive false alarms.

FDA should publish clear guidance on acceptable statistical methods for GenAIenabled medical devices, including sample-size justification, confidence levels,
subgroup analysis, repeated-output testing and postmarket alert thresholds.
Alternatively, FDA should identify credible consensus standards or reference
documents that manufacturers can use to understand what statistical evidence the
Agency considers acceptable.

13. Where is synthetic data appropriate or inappropriate, and what safeguards are needed?

Synthetic data should not be used as evidence that a device is safe, effective or
performs as intended. Its use should be limited to development activities, such as
exploratory testing, adversarial challenges, rare-case enrichment and the design of
test scenarios. Although synthetic data can help developers identify potential
weaknesses and exercise unusual conditions, it cannot replace independent,
representative real-world data for verification or validation.
 Verification and validation should use independent, representative real-world data
that reflect the intended patient population, clinical setting, users, workflows and
reasonably foreseeable conditions of use. Synthetic data cannot replace real data for
verification or validation because synthetic generators may reproduce existing
biases, omit clinically important variability and fail to capture realistic failure
modes. Development datasets—including synthetic test fixtures—should remain
separate from held-out real-world verification and validation datasets.

Ongoing monitoring of device performance, safety, bias and drift should also be
based on real-world data. Manufacturers should establish a representative baseline
and use actual post-deployment data to identify changes in performance across
patient populations, sites, users and clinical conditions. Synthetic data may
supplement investigations or help reproduce a suspected failure, but it should not
be used to demonstrate continued performance or to obscure deterioration
observed in real-world use. Regardless of the number of generated cases, synthetic
data do not establish clinical representativeness or replace real-world evidence.

14. How should comparators and acceptance criteria be selected for open-ended outputs?

Open-ended responses should be judged on whether they are clinically correct and
useful, not on whether they match a single preferred answer word for word. The
evaluation criteria should reflect the specific task and identify the information that
must be included, statements that would be unsafe or unacceptable, important
omissions, appropriate expressions of uncertainty, escalation requirements and the
accuracy of any citations.

Different wording may be equally acceptable, but clinically significant errors—such
as inventing a disease stage, overlooking a contraindication or failing to recommend
urgent escalation—should not be masked by a strong overall score.

Qualified reviewers should be clinically competent in the relevant medical specialty
and independent of the manufacturer. They should assess the outputs using a
consistent rubric, with agreement between reviewers measured and disagreements
resolved through adjudication. Serious failure types should have their own
acceptance limits. Performance should also be compared with relevant clinical
practice or an accepted reference standard. If an automated scoring tool is used, the
manufacturer should validate it against qualified human reviewers and continue to
monitor its reliability.

15. Should performance be assessed relative to likely care in the absence of the device?

Performance relative to likely care in the absence of the device is particularly
important for high-risk devices and devices used in urgent scenarios involving
critically ill patients, where an incorrect, misleading or delayed output could result
in serious harm. In those settings, evidence should address clinically meaningful
 worst-case outcomes and demonstrate that the device does not introduce
unacceptable delay, automation bias or inappropriate reliance compared with
expected care. Evaluate both standalone model performance and the combined
human-device system. For example, a triage assistant may reduce unnecessary visits
while increasing missed urgent cases or delaying escalation; all material benefits
and harms should be reported. Manufacturers should also monitor whether user
behavior, reliance on the device and the applicable standard of care change after
deployment.

16. Should independent third parties participate in benchmarking and clinical
confirmation?

Independent third parties can improve the credibility of benchmarking and clinical
confirmation, particularly for high-risk or autonomous devices, and for devices used
in urgent scenarios involving critically ill patients, where an incorrect or delayed
output could result in serious harm. Third-party evaluators should be appropriately
accredited or otherwise demonstrably qualified for the relevant technical and
clinical scope. Their independence and potential conflicts of interest should be
assessed before engagement and monitored throughout the relationship, with
documented controls addressing financial interests, prior development
involvement, access to proprietary information, evaluator rotation where
appropriate, confidentiality and reproducibility.

Oversight and testing should be risk based: higher-risk devices and functions should
receive more stringent, frequent and independent challenge testing, while lowerrisk devices should still meet a defined baseline level of independent evaluation
proportionate to their intended use and reasonably foreseeable harms. The third
party should confirm that the manufacturer has clearly defined the scope and
boundaries of the AI-enabled device—including the model, prompts, retrieval
sources, tools, interfaces, intended users and clinical workflow—and should assess
whether the proposed validation methods, datasets, comparators, acceptance
criteria and statistical approaches are appropriate for that scope. Activities may
include locked challenge datasets, blinded clinical adjudication, cybersecurity
testing, statistical audits, undisclosed severe conversational scenarios and periodic
field-performance review. Independent review strengthens the evidence but does
not transfer the manufacturer’s responsibility for evaluator qualification and
oversight, device design, risk controls, corrective action or regulatory decisions.

17. Can the competency-based approach be applied across different model architectures?

Clinical competency domains can be architecture-neutral, but test methods must
address architecture-specific hazards. Retrieval-augmented systems need retrieval
and citation checks; multimodal systems need cross-modal consistency; fine-tuned
models need training-data lineage; agentic systems need tool-permission and
transaction tests. Pin the complete configuration, including model, prompts, tools,
 retrieval corpus and interface. Verify the full device function, not merely the
foundation model. Changes in any component can require targeted revalidation.

18. When is greater reliance on postmarket monitoring appropriate?

Greater reliance on postmarket monitoring may be appropriate when the
manufacturer has already established a sufficient premarket evidence base through
bench testing, simulation, software verification and validation, clinical or humanfactors evaluation as applicable, and validation of relevant risk controls. It is most
supportable for a lower-risk device or function where reasonably foreseeable
failures are associated with low-severity harms, outputs remain reviewable or
reversible, and effective downstream safeguards are available. Postmarket
monitoring may then be used to address residual uncertainty that cannot practically
be eliminated before launch; it should not compensate for unmitigated unacceptable
risk or missing essential premarket evidence.

Before deployment, the manufacturer should establish a validated performance
baseline, bounded intended use, predefined monitoring endpoints, safety controls,
staged-rollout criteria and rollback or suspension thresholds. The postmarket plan
should be proportionate to the device risk and should specify who reviews alerts,
investigation timelines, patient protections and escalation requirements. For
example, a lower-risk clinical assistant may progress from silent-mode evaluation to
supervised use only after predefined safety gates are satisfied and monitoring
confirms that performance remains within validated limits. Greater reliance on
postmarket evidence should therefore complement, rather than defer or replace,
appropriate premarket testing and validation.

19. What postmarket monitoring approaches should FDA consider?

Manufacturers should implement a risk-based AI performance monitoring plan
integrated with PMS, complaints, CAPA and change control.

Track clinically significant error rates, subgroup outcomes, drift, hallucinations,
unsupported citations, escalation failures, user overrides, near misses, version
changes and reported harms.

High risk AI devices should have performance metrics be monitored with a
combination of clinical evidence (i.e., surveys, questionnaires, retrospective log file
reviews, registry studies) and continued AI performance validations.

20. Can machine-based supervisory agents support postmarket monitoring?

Supervisory AI agents can triage outputs, flag unusual conversations, check citation
grounding and identify prohibited actions.
 They should not be assumed independent when they share a foundation model or
training biases with the primary device. Combine validated rule-based constraints,
diverse detection methods and qualified human review. A monitor could flag a
chest-pain conversation that ends without urgent escalation, while deterministic
controls prevent an unauthorized prescription order. Supervisory agents should
themselves be validated for their intended monitoring function and undergo their
own documented assessments of performance, drift and bias. Their updates should
be controlled through the manufacturer’s quality system. The manufacturer remains
accountable for the design, validation, deployment and ongoing performance of the
supervisory agent; using an automated monitor does not transfer or reduce that
responsibility.

21. What roles should stakeholders play in postmarket monitoring?

The manufacturer owns device safety, risk management, supplier oversight,
performance monitoring and corrective action throughout the lifecycle.

Foundation-model suppliers should provide version histories, known limitations,
incident notices and change information through quality agreements.

Healthcare organizations and clinicians contribute workflow evaluation, adverseevent reporting and clinical adjudication.

Patients need accessible feedback channels and clear limitations.

Independent assessors can audit evidence of each of the stakeholders.

22. How should post-deployment modifications be re-evaluated?

Manufacturers should manage each post-deployment modification through their ISO
13485 quality management system and established design and change-control
procedures. A proposed change to model weights, prompts, system instructions,
retrieval sources, tools, guardrails, user interface, hosting, third-party components
or clinical workflow should be initiated as a formal change-control request rather
than implemented as an informal software update. The request should describe the
proposed modification, its rationale and source, the affected device configurations
and users, the intended implementation date, and the responsible functions.

The change-control review should include a documented, cross-functional impact
assessment covering:

• intended use and claims;
• device specifications and software requirements;
• hazards and risk-control measures;
• clinical performance;
 • usability;
• cybersecurity;
• data and dataset suitability;
• supplier dependencies;
• labeling and training;
• interoperability; and
• previously approved verification and validation evidence.

The assessment should determine whether the modification introduces a new
hazard, changes the probability or severity of an existing harm, affects a safetycritical control, or requires updates to the risk-management file, technical
documentation, software configuration records or post-market surveillance plan.

Based on the assessed impact, the manufacturer should define and approve a
proportionate verification and validation plan before release. This may include
targeted regression testing, mandatory safety-critical sentinel tests, subgroup and
multi-turn conversational evaluations, human-factors testing, clinical revalidation
and confirmation that established acceptance criteria remain satisfied. For example,
a prompt wording change that alters the directiveness of an emergency referral may
require renewed usability and clinical evaluation even if the underlying model
weights are unchanged.

The change-control record should document reviewers and approvals, test results,
unresolved anomalies, configuration and model identifiers, dataset and prompt
versions, deployment controls, training needs, rollback assets and the regulatory
assessment. It should also determine whether the modification is covered by an
authorized Predetermined Change Control Plan or triggers an applicable premarket
submission, notification or other regulatory obligation.

After implementation, the manufacturer should confirm correct deployment,
monitor defined post-release indicators and complaints, compare performance with
the prior version, and initiate corrective and preventive action or rollback when
acceptance thresholds are exceeded. The request should be closed only after
required documentation has been updated and objective evidence demonstrates
that the modified device remains safe, effective and compliant.

23. How should PCCP and change-control concepts be adapted for GenAI-enabled devices?

A Predetermined Change Control Plan should translate the general PCCP elements—
a Description of Modifications, Modification Protocol and Impact Assessment—into
a clearly bounded GenAI configuration envelope. The plan should identify the
specific changes that may be implemented without a new marketing submission, the
rationale for anticipating them, the risks they may introduce and the evidence
required before release. The permitted change should remain consistent with the
 device’s intended use, indications, user population, level of autonomy and essential
safety characteristics.

For GenAI-enabled devices, the scope should extend beyond model weights. It may
need to cover system and developer prompts, retrieval sources and indexing
methods, tool permissions, guardrails, orchestration logic, decoding parameters,
user-interface controls and the version of any third-party foundation model. Each
change category should have predefined eligibility criteria, prohibited boundaries
and a validation protocol. Testing should address not only average task
performance, but also clinically significant failure modes, factual grounding and
citation accuracy, inappropriate certainty, refusal and escalation behavior, multiturn interactions, subgroup performance, cybersecurity, human factors and
regression of established risk controls.

The PCCP should also define release gates, independent approval where
appropriate, versioned configuration lineage, post-deployment monitoring
thresholds, complaint and incident review, and rapid rollback or disablement
procedures. For example, a controlled retrieval-index refresh may fit a validated
protocol if source eligibility, provenance, update frequency, citation integrity and
sentinel clinical questions are prespecified. By contrast, adding an unvalidated
external tool, materially increasing autonomy, changing the intended clinical role or
allowing unconstrained online learning should ordinarily fall outside the plan and
undergo the usual regulatory change assessment. A PCCP should therefore enable
disciplined iteration within an approved evidence-based boundary; it should not
function as unlimited authorization for self-modifying AI.

24. How should manufacturers manage changes initiated by third-party foundation model
developers?

Treat supplier model updates as foreseeable supply-chain hazards. Contracts should
require material-change notice, stable version identifiers, evaluation information,
incident reporting and access to pinned versions or rollback where feasible.

A controlled gateway should prevent silent substitutions.

Following supplier changes, repeat clinically critical tests for escalation, refusals,
prompt injection, citations, tool use and subgroup performance. Maintain outage
and model-retirement contingency plans. If supplier transparency is inadequate,
assess the residual uncertainty against intended clinical use.

Outsourcing a foundation model does not outsource manufacturer responsibility.

A supplier-initiated model change should be opened as a formal change-control
request and should not be deployed until the manufacturer completes the required
impact assessment, verification, validation, approval, and regulatory assessment.
 The manufacturer should also determine whether the change falls within an
authorized PCCP.

25. Would voluntary Foundation Model Master Files be practical and useful?

Voluntary Foundation Model Master Files may reduce duplicate technical
submissions if they provide reliable model identity, version lineage, evaluation
scope, limitations, data governance, cybersecurity and change notifications. They
cannot establish the safety or effectiveness of every downstream device.

Each manufacturer must validate its own intended use, prompts, retrieval, tools,
interface and clinical workflow. FDA should define authorization to reference such
files, confidentiality, updates and what happens when evidence becomes stale.
Supplier evidence is useful input, not a substitute for device-level verification.

26. What additional considerations apply to agentic GenAI-enabled devices?

Agentic systems can plan, call tools, alter records and execute actions across
multiple steps.

Risk analysis must cover:

• permission escalation,
• tool misuse,
• prompt injection,
• context loss,
• action chaining,
• irreversible operations and
• cascading errors.

Apply least-privilege tools, allowlists, authenticated authorization, deterministic
preconditions, transaction limits, human approval for consequential actions and
immutable audit traces.

For example, an agent may draft a clinical order but should not execute it without a
validated authorization and review pathway. Test multi-step adversarial scenarios,
interrupted workflows, rollback, data leakage and conflicting instructions.
Continuously monitor action success, prohibited attempts, near misses and clinical
impact. A supervisory agent can assist, but critical safeguards should not rely solely
on another probabilistic model.