Doctors for America

See attached file(s)

Dockets Management Staff (HFA-305), Food and Drug Administration, 5630 Fishers Lane, Rm.
1061, Rockville, MD 20852

Re: Docket No. FDA-2026-N-7874 — Considerations for the Regulation of Generative
AI-Enabled Medical Devices: Discussion Paper and Request for Feedback

To the Center for Devices and Radiological Health, Digital Health Center of Excellence:

Doctors for America (DFA) supports the Center's effort to build a regulatory approach for
generative AI (GenAI)-enabled devices, but urges that reasonable assurance of safety and
effectiveness — not speed to market — remain the organizing principle. The discussion paper
correctly identifies two central challenges: applying a risk-based approach to regulatory
requirements and determining how to validate scientific evidence. Several proposed solutions,
however, would shift the evidentiary burden from manufacturers to postmarket systems, and
thus to clinicians and patients, who lack the data and background needed to detect these risks.

DFA is an independent organization of more than 42,000 physicians and trainees advocating for
policies that serve our patients' health and access to high-quality care. DFA accepts no funding
from pharmaceutical or medical device companies. Our FDA Task Force, a cross-specialty
group of physicians and trainees, works to ensure patients have timely access to medical
products that are truly safe and effective. These comments build on our November 2025
comments to the Digital Health Advisory Committee (DHAC) on GenAI-enabled digital mental
health devices [1] and on our ongoing assessment of how this rapidly advancing technology
affects patient care.

Summary of recommendations

The FDA Task Force of Doctors for America is committed to supporting FDA in its core mission:
authorizing medical products that are proven safe and effective. The need for rigorous validation
and the desire to bring innovative products to market are in dynamic tension, and virtually all
authorized products carry some degree of risk. It is our position that premarket review should be
as robust as possible, with the ultimate decision free from both industry and political influence.
Because no system of review can anticipate all consequences, an equally robust postmarket
surveillance system is an absolute requirement.

The Task Force recognizes that GenAI has great potential to improve health care, but recent
reports of unanticipated consequences bring us back to present realities and to our obligation to
protect patients from harm.
Protecting patients requires a regulatory framework that serves not only current care but the
near and far future. That means closing a major regulatory gap, leaving room for advances in
artificial intelligence (AI) while providing the guardrails necessary to prevent harm. We also
recognize that congressional action will be needed to create an enforceable set of rules for the
use of AI in health care.

Our recommendations:

1.​ Keep premarket evidence proportionate to risk, with a floor. Postmarket monitoring
should never substitute for premarket clinical evidence when errors are irreversible,
time-critical or patient-facing.
2.​ Treat benchmark "competency" as a surrogate, not an endpoint. The proposed
two-part demonstration of clinical competency, that is, scalable non-clinical benchmarks
followed by clinical confirmation, addresses the range of inputs and outputs, but it is
untried. Like surrogate markers in drug development, benchmark results will correlate
with clinical outcomes to varying degrees, and that correlation deserves critical scrutiny
and continued improvement.
3.​ Expand the risk framework to include error detectability, reversibility, scale of exposure
and user vulnerability (including children and people with mental illness). Relational
features that build reliance on a tool are a risk factor and should be treated as a
separate component of the risk evaluation.
4.​ Require, not invite, independent postmarket surveillance for action-directing and
action-taking functions, with defined triggers for re-evaluation and a human-controlled
ability to suspend a device.
5.​ Make transparency mandatory where it matters for safety. Labeling, subgroup
performance, version disclosure and foundation model information should not depend on
voluntary master files that shield information from analysis outside FDA.
6.​ Address the gaps: patient privacy outside HIPAA, bias in foundation model training
data, allocation of liability, and safeguards against disproportionate industry influence on
the framework itself.

Sections I–III: Introduction, background and definitions

The paper's framing should give equal weight to patient protection. The stated goal is a "nimble"
approach that applies least burdensome principles [2]. DFA recognizes the need for adaptability
in the face of rapidly changing technology, but the least burdensome principle is open to broad
interpretation that could compromise the assurance of safety and effectiveness. We ask that any
guidance clarify what "nimble" means, so that it is not read as permission to accept weaker
evidence for novel, poorly understood technology.

Clinical impact depends on real-world use, not on capability. GenAI may expand access,
reduce documentation burden and bring specialty knowledge to underserved settings. Our 2025
comments noted that 169 million Americans live in mental health care shortage areas, and that
well-validated tools could help fill that gap [1]. Yet the record for AI devices already on the
market is sobering. Devices without clinical validation were recalled more often and in larger
numbers, and more than 40% of AI device recalls occurred within the first year after clearance
[3]. GenAI adds open-ended inputs, variable outputs and dependence on third-party models to
that baseline. Clinical benefit should be demonstrated, not assumed.

The device boundary will be the largest source of unregulated risk. The paper notes that
FDA regulates devices, not GenAI as such, and that many software functions fall outside the
device definition or under enforcement discretion. In practice, the GenAI tools patients and
clinicians use most — general-purpose chatbots, ambient scribes, "wellness" and companion
apps — sit largely outside the framework. The November 2024 DHAC executive summary itself
acknowledged that a scribe intended only to summarize can insert a diagnosis that was never
discussed [4]. In its feedback on the discussion paper, Stanford's Brainstorm Lab found that the
products most harmful to adolescents would score as low-risk, non-directive tools on the
proposed grid [5].

The shifting line between wellness products and medical devices is illustrated most clearly by
WHOOP, Inc.'s Blood Pressure Insights (BPI) feature. Under FDA's general wellness policy as it
stood in 2019, the feature appeared to qualify as a low-risk wellness product. In 2025, FDA told
WHOOP in a warning letter that BPI was an unapproved medical device because blood
pressure readings are used to diagnose hypertension. In 2026, FDA revised its wellness policy
to treat non-invasive products that estimate or output physiological parameters as wellness
products, and WHOOP's feature was recategorized without any public change to the product [6].
Without a clear, common-sense definition, neither the public nor physicians will know when a
product should be FDA-authorized, nor be able to alert FDA when a product's uses have
expanded into regulated territory.

DFA recommends that FDA:

●​ define intended use by reasonably foreseeable use, not labeled indication alone, for
products that may predictably drift into diagnosis or treatment;
●​ publish criteria for when a product moves from wellness or non-device status into device
status, and how FDA will act when that happens; and
●​ identify plainly, in any future guidance, where existing statutory authority is insufficient,
so Congress can act. The paper's decision not to address legal authority should not
leave the public without a clear account of the gaps.

Section IV: Assessment of risk (Questions 1–6)

The two-axis framework is a sound starting point, but the two axes as they stand
inadequate to capture patient safety risk.

Additional dimensions (Q1). The two proposed axes, namely (1) device activity, ranging from
purely informational to fully autonomous; and (2) consequences of error do capture important
differences between products. But a two-axis construct cannot capture the multidimensional
nature of GenAI or the differing vulnerabilities of diverse populations. DFA supports adding each
dimension FDA suggests: reversibility, downstream safeguards, time pressure and traceability to
primary sources; and we suggest considering other factors, including but not limited to:

●​ Detectability of error. A plausible, authoritative-sounding error that the user cannot
verify is more dangerous than an obvious one. This applies well beyond measurement
functions.
●​ Scale of exposure. A clinician's error is local; a model's error repeats across every
patient, site and device built on the same foundation model, as the experience with
IBM's Watson for Oncology illustrates [7]. A small per-interaction risk can become a
population-level harm.
●​ User vulnerability. Children, older adults with cognitive impairment, and people with
psychosis, suicidality or eating disorders are less able to evaluate outputs and more
susceptible to harm. Our 2025 comments documented risks of impaired reality testing
and dependency in these groups [1]; the Group for the Advancement of Psychiatry's
Committee on Aging has made similar points for older adults [8].
●​ Relational features. Persistent memory, anthropomorphic voices or avatars, proactive
notifications and affective language change how users rely on a device. These features
can pose significant risks, especially in multi-turn use [9].
●​ Frequency and opportunity for intervention. FDA should also weigh how often
harmful failures occur, how many patients may be exposed, and whether there is a
realistic opportunity for human intervention before harm occurs.

Reversibility deserves particular weight. An incorrect informational explanation that is caught
before a clinical decision presents a different level of risk from an incorrect medication
recommendation that is acted on and administered. Likewise, an error that is easily identified
and reversed differs from one that produces a delayed diagnosis, a missed emergency or
irreversible harm.

Consider an AI dosing tool in the electronic health record (EHR) that suggests too high a dose.
If the error is flagged prominently, before the order is signed, with clear and urgent language, a
clinician is likely to catch it. If the same suggestion is embedded in the order with vague, easily
bypassed wording, the identical error carries far greater risk. Once the drug is given, the chance
to intervene may be gone and the harm may not be reversible. Detectability, opportunity and
time for intervention, and reversibility often determine whether an error remains a correctable
mistake or becomes a patient safety event.

Directiveness (Q2). DFA agrees with FDA that directiveness depends on the substance and
context of an output, not on particular words such as "consider" or on a "talk to your doctor"
disclaimer. Risk should be based on what the output is likely to cause the user to do.

The lisinopril example posed by the discussion paper is useful for illustrating the continuum of
directiveness, but medical treatment is rarely so straightforward. We suggest converting the
four-step continuum into a formal scale, classified by observed behavior rather than by the
intended-use statement. A device should be tested at the highest level of directiveness of which
it is capable. This also addresses multi-turn use, where outputs can drift toward directiveness as
a conversation continues.

Specificity and personalization are the strongest modifiers, because personalized suggestions
are more likely than general information to replace a user's own judgment. Specific AI
suggestions can sway even experienced physicians [10,11].

Chest pain illustrates the point. A general response explaining that chest pain has many cardiac
and non-cardiac causes is non-directive. A system that incorporates age, symptoms and history
and concludes that the pain is musculoskeletal and can be monitored at home has directed the
patient toward a course of action. A disclaimer such as "consult a physician if you are
concerned" should not make that output low risk. Conversely, a response that identifies
concerning features and recommends immediate evaluation is explicitly action-directing and
should be evaluated according to the consequences of that recommendation. Testing should
include realistic scenarios in which users progressively provide personal information and ask
increasingly specific questions.

Patient-facing functions (Q3). DFA agrees with the distinction CDRH draws between
patient-facing and health care provider (HCP)-facing informational functions. Patient-facing
functions should be treated as higher risk by default. Patients often describe symptoms in ways
that require a trained clinician to interpret, which can produce false reassurance in some cases
and false alarm in others. Many patients will use these tools instead of care, not alongside it.
DFA supports patient access to information, but protections must be built into devices. The
appropriate safeguards are design requirements rather than disclaimers:

●​ A model's stand-alone accuracy does not predict how safe it is when patients use it [12],
as the discussion paper recognizes (Section E.4, p. 25). Devices should be validated
with real users in each subgroup likely to use them.
●​ Because patients (and even clinicians) may not know what information is needed, a
structured symptom intake should collect the clinically necessary facts before giving
advice.
●​ A clinician must be in the loop for high-risk functions. In a randomized trial of a GenAI
therapy chatbot, staff had to intervene for both safety concerns and inappropriate
responses [13]. Given the known occurrence of automation bias, testing must confirm
that reviewers actually catch errors [10,11].

Generalist and specialist users (Q4). The distinction is relevant, but FDA should not assume
generalists are less capable of using AI-generated medical information. Especially in rural and
underserved areas, generalists routinely manage complex, multisystem illness. Insulin
management for patients with insulin-treated diabetes, for example, is routinely handled by
primary care physicians even where endocrinologists are available. The key question is whether
the intended user can understand, evaluate and appropriately act on the output in the
circumstances in which the product is used.
Multi-turn conversations (Q5). Intended use should be characterized by a conversation's
plausible endpoint, not its opening turn. "Multi-turn" must mean days and months, not three
exchanges; a conversation that begins with general education may evolve into personalized
diagnostic, treatment or triage advice. In testing of widely used chatbots, safeguards eroded as
conversations lengthened and did not persist across new sessions [14]. Evaluation should be
stratified by conversation length and cover session boundaries. Manufacturers should
demonstrate that the product remains within its intended use throughout the interaction and
identify circumstances in which it is likely to become more directive or give higher-risk advice.

Escalation errors (Q6). DFA agrees that under-escalation and over-escalation should be
measured and reported separately and never blended into a single accuracy figure [15]. The
harms are not equivalent. Returning to the chest pain example, imagine a chatbot incorrectly
reassuring a patient that the symptoms are likely due to indigestion. The error may be hard to
detect because the response sounds authoritative, no clinician may be available to review it,
and the harm becomes progressively less reversible as treatment for an acute coronary
syndrome is delayed. A system that incorrectly recommends emergency evaluation for
indigestion creates cost, anxiety and inconvenience, but that harm is generally reversible.

Failure to recognize an emergency can lead to delayed care, further harm or death, while
over-escalation can lead to unneeded testing, patient anxiety and misuse of resources. We
therefore agree with the American College of Allergy, Asthma and Immunology [16] and the
Stanford Lab for Mental Health Innovation [5] that acceptance criteria may need to be
intentionally asymmetric, with very low tolerance for under-escalation in conditions such as
suicidality, chest pain, stroke or anaphylaxis.

DFA also endorses a third category: escalating badly. Sponsors should demonstrate, end to
end, what happens after the system determines that escalation is necessary: who receives the
alert, how quickly they are expected to act, whether the patient successfully reaches an
appropriate professional, and what happens outside normal business hours.

Human oversight should mitigate regulatory risk only when it is shown to work in the real-world
conditions of use. FDA should evaluate not only overall accuracy but the severity, frequency,
detectability and reversibility of errors and the opportunity for intervention before harm. Patients
and clinicians should have a meaningful role in defining acceptable trade-offs between underand over-escalation; these should not be set solely by manufacturers or by aggregate
performance metrics.

Section V: Competency-based premarket evaluation (Questions 7–17)

Benchmarking plus clinical confirmation is a useful structure, but competency must not
become the GenAI equivalent of a surrogate endpoint. Real-world effectiveness and
safety should remain paramount.
The framework (Q7). Passing a sophisticated benchmark shows that a device performs well on
that benchmark; it does not show that deploying the device improves patient care or clinical
outcomes.

Evaluations of current models reveal significant deficiencies at every step from benchmark to
user. About 95% of published LLM evaluations did not use real patient care data, and few
measured fairness, bias or toxicity [17]. Models that score highly on medical licensing exams fail
on real cases and perform significantly worse at diagnosis than physicians [18]. Members of the
public using a high-scoring model did no better than controls [12], and physicians given an LLM
did not improve their diagnostic reasoning even though the model alone outperformed them
[19]. Human implementation adds further risk: clinicians follow incorrect AI suggestions [11].
Each step from benchmark to user adds decision and outcome uncertainty.

We expect competency to improve, but current performance gaps argue against broad
deployment, and if today's flaws are not rigorously evaluated, they will persist into future
versions. FDA should therefore require benchmarking to be clearly defined, scientifically valid,
and appropriate to the device's intended use and risk profile. Benchmarking should be one
component of a broader evidence framework, not a substitute for clinical validation.
Manufacturers should demonstrate that benchmark performance translates into safe and
effective performance under realistic conditions, including evaluation of clinically meaningful
errors, subgroup differences and the effects of human-AI interaction. FDA should also require a
defined process for ongoing reassessment as evidence accumulates about real-world use and
patient outcomes, so that evaluation standards evolve with the technology and changes in
model capability do not outpace the evidence needed for safe deployment.

Scaling evidence to risk (Q8). DFA agrees that the two-axis framework, augmented as
described above, should set the level of premarket evidence. It should scale the breadth of
benchmarking and the level of clinical confirmation independently. Strong benchmark results
should never move a high-risk device to a lower tier of clinical evidence. The required evidence
should also reflect error detectability, reversibility, opportunity for intervention, time pressure,
scale of exposure and the effectiveness of downstream safeguards. DFA proposes the following
tiers:

Risk tier Activity / Benchmarking Clinical confirmation
consequence

Low Non-directive, Core elements, standard test Retrospective evaluation
clinician-facing; conditions
 minor, reversible
consequences

Moderate Action-directing, Full elements, including Independent clinician
clinician-supervised; atypical presentations and adjudication or shadow
moderate subgroups deployment
consequences

High Patient-facing Full elements plus pass/fail Real-world validation in
action-directing, or floors for named the intended population
time-critical high-consequence failures;
escalation; serious multi-turn and
consequences session-boundary testing

Highest Autonomous All of the above plus long Prospective comparative
action-taking; task-chain and adversarial evidence, randomized
irreversible harm testing where feasible, plus
possible intensive postmarket
surveillance

These tiers should be a guideline, not a rigid framework. A clinician-facing decision-support tool
may appear lower risk because a physician is expected to review its recommendations, yet as
such tools become routine, clinicians are prone to automation bias [10,11].

An insulin-adjustment tool presented as clinician-facing and non-directive may still warrant
stronger evidence if an incorrect recommendation could cause severe, irreversible
hypoglycemia. FDA should consider whether the clinician has sufficient time, access to relevant
clinical information and authority to challenge or override the recommendation. Manufacturers
should demonstrate through realistic workflow testing that clinicians reliably identify and correct
dangerous errors, rather than assuming that human oversight will prevent harm.

As our 2025 comments urged, the De Novo pathway will not suit every case; FDA should use
premarket approval where devices will be used in high-risk populations [1].

Benchmark elements (Q9). The Appendix A elements are thoughtful. DFA recommends
adding the following:

●​ Omission and harmful affirmation. Beyond incorrect answers, evaluation should
capture failures to identify important findings, ask necessary follow-up questions or
 recognize situations requiring escalation. Research from Stanford found that the most
serious failures were missed questions and agreement that reinforced a condition, such
as reassurance-seeking in obsessive-compulsive disorder [9]. Sycophancy, the tendency
to validate a user's thoughts, should be a named and explicitly tested failure mode where
relevant to the product [1].
●​ Atypical and non-textbook presentations as a distinct element. Atypical presentations
are common in medicine: myocardial infarction without chest pain, for example, as are
disorganized, non-textbook histories from patients. LLMs perform worse when cases
depart from the textbook pattern [20].
●​ Longitudinal and chronic disease reasoning, which requires different thought
processes and information gathering and assessment that is distinct from acute illness.
Benchmarks test single-encounter diagnosis, but much GenAI use will involve managing
chronic conditions over time. That requires finding relevant facts scattered across long
records, where models perform poorly [21]; recognizing what is missing [22]; and
reconciling guidelines that conflict in patients with multiple conditions [23]. Errors in
chronic care are often silent and cumulative, so premarket testing should include
multi-visit cases with changing data, and postmarket monitoring should not rely on
adverse event reports alone.
●​ Separate pass/fail floors for named high-consequence failures. For products with
high-consequence failure modes, FDA should establish prespecified criteria for clinically
significant failures, such as missed suicide risk, missed sepsis or an unsafe insulin
adjustment. These failures should not be obscured by strong performance on unrelated
tasks.
●​ Age- and developmental-stage-stratified testing whenever children are in the
intended population, rather than a single "pediatric" subgroup [24]. Communication,
symptom reporting, medication dosing, safeguarding and the ability to recognize risk
differ substantially among infants, young children, adolescents and adults.
●​ Reversibility. Testing should establish whether users can recognize an incorrect output,
whether an error can be corrected before harm occurs, and whether the intended
workflow provides sufficient time and information for intervention.

Benchmark validity and independence (Q10). Benchmark validity should be shown through
studies linking benchmark results to independently observed clinical outcomes, rather than
inferred from benchmark scores alone.

Sponsor-developed benchmarks should not be the primary basis for authorization.
Manufacturer-sponsored drug and device studies more often report favorable results and
conclusions than independently funded studies [25], and LLMs can memorize or be optimized
against benchmark content, inflating scores that collapse under modified testing [26].
Historically, AI device evaluations have been mostly retrospective and single-site [27]. DFA
recommends a layered approach. FDA should set evaluation standards and qualify benchmarks
through its Medical Device Development Tools program, with accredited independent
evaluators, including professional societies and assurance laboratories, meeting strict
independence and conflict-of-interest criteria. Vetted independent reviewers should hold
sequestered, versioned and regularly refreshed test sets drawn from multiple sites and
representative populations, and FDA should maintain its own sequestered test sets for the
highest-risk functions. Sponsor-developed tests may supplement this evidence only with
independent validation of items and scoring. All test results, rubrics and adjudication methods
should be public.

DFA recommends the following testing conditions:

●​ Sequestered items: held by a party other than the sponsor and never exposed during
development.
●​ Versioned and refreshed: Static, public benchmarks saturate as models improve. Once
leading models score near the ceiling, a benchmark no longer distinguishes safe from
unsafe performance or detects the rare, high-consequence failures that matter most.
Benchmarks used for authorization should be refreshed regularly and include difficult,
held-out items on which current models do not score near the maximum.
●​ Documented provenance: data source, population, question source and derivation,
and partitioning method.
●​ Representative and multi-site: representativeness should be demonstrated, not
assumed. Rural, safety-net and underserved populations must be adequately
represented.
●​ Real patient data: not only exam-style questions.
●​ Real human users with multi-turn interaction.
●​ Prespecified acceptance criteria: as discussed above.
●​ Independent clinical adjudicators with full conflict-of-interest (COI) disclosures.
●​ Published results and scoring rubrics: integral to appropriate clinical use, these must
not be kept confidential between sponsor and evaluator.

Sepsis recognition tools already integrated into some EHR platforms illustrate the need. These
tools may perform well on classic presentations but miss patients with atypical symptoms,
incomplete laboratory data or less common manifestations. Independent testing should examine
whether a benchmark captures these presentations and whether results predict safe
performance in the intended setting. Benchmark results should inform, but not by themselves
determine, the conclusion that a device is safe and effective.

Clinical confirmation (Q11). DFA supports scaling clinical evidence to risk, but proportionality
should not mean that each sponsor decides for itself what evidence is sufficient. DFA
recommends that FDA define triggers that create a presumption that a prospective clinical study
is required, rebuttable only with a published justification. A prospective study is needed when
retrospective data and benchmarks cannot capture the most likely source of harm. Any one of
the following should trigger the presumption:

●​ High-impact category: the function falls in Category III or IV of the International Medical
Device Regulators Forum (IMDRF) software risk framework — software that treats or
diagnoses a serious or critical condition, or drives clinical management of a critical
condition [28].
 ●​ Autonomous action: the device acts without real-time clinician review. FDA authorized
the first autonomous AI diagnostic on the basis of a prospective pivotal trial [29].
●​ Effect depends on human interaction: the device is patient-facing or conversational.
In a randomized study, LLMs performed well alone but gave users no advantage over
controls, and testing with simulated users did not predict the failures [12].
●​ Time-critical escalation or irreversible harm: a consumer GenAI triage tool
under-triaged 52% of physician-confirmed emergencies in structured testing [30].
●​ Low detectability of errors: the intended user cannot independently verify the output.
●​ Deployment differs from development data: a widely deployed sepsis model
performed substantially worse on external validation than its developer reported and
missed two-thirds of sepsis cases [31].
●​ Vulnerable populations with sparse data, including children, people in psychiatric
crisis and people with cognitive impairment.
●​ Long-term use, where harms such as dependency or eroding safeguards accumulate
over weeks or months.
●​ Claims of improved outcomes, access or efficiency, which can be demonstrated only
prospectively.

The type of prospective study should follow the trigger. A prospective observational study,
including shadow deployment, may suffice when the concern is real-world performance and
outputs do not yet reach patients. A prospective interventional study, randomized where
feasible, is needed when the concern is how users act on outputs or when the device acts
autonomously. A device that meets none of the triggers may rely on benchmarking with
retrospective or clinician-adjudicated confirmation. For pediatric and other vulnerable
populations, our 2025 comments also recommended sandbox-type testing before clinical
studies, consistent human oversight and longer-term follow-up [1].

Statistics and synthetic data (Q12–Q13). Real patient data from the intended population
should be required for clinical confirmation; synthetic data may supplement it but not replace it.
Models trained on generated data progressively lose the tails of the original distribution [32] —
and the tails are where rare conditions, atypical presentations and underrepresented groups
appear. Simulated patients are more cooperative and forthcoming than real ones, and
simulation has failed to predict real-user failures [12]. Synthetic data is best used for
stress-testing [33], with its origin declared and every subgroup also checked against an
independent real-world reference set.

We recognize that synthetic data is a valuable supplement to real-world evidence especially in
rare cases/conditions or with underrepresented populations, but it should not substitute for
clinical data.

Synthetic data can reproduce the limitations and biases of the models or datasets used to
generate it. If the model generating synthetic cases systematically underrepresents atypical
presentations, the resulting evaluation dataset may fail to detect the same shortcomings in the
device under review.
Imagine a manufacturer develops a generative AI tool to recognize a pulmonary embolism.
Synthetic data would be valuable in creating large data models, variations in age, symptoms,
risks and other presentations that are atypical.

However, if the synthetic cases are generated by a model that disproportionately represents
classic presentations, they may not adequately capture the subtle or atypical symptoms
encountered in older adults, pregnant patients, or patients with multiple comorbidities.

A system could therefore perform well on the synthetic dataset while remaining less reliable in
the very populations for whom additional testing is needed.

DFA urges that sponsors be required to:

●​ Distinguish data sources explicitly: tell us which testing cases utilize simulated inputs
versus genuine patient records.
●​ Disclose generation methodology: detail the precise mechanisms by which artificial
cohorts were constructed and verified.
●​ Engage independent medical reviewers to confirm that non-real-world scenarios
maintain clinical authenticity and properly mirror target demographics.
●​ Cross-validate datasets against authentic health trends and established clinical
baselines.
●​ Prohibit sole reliance on artificial data for substantiating safety in high-consequence
interventions.
●​ Execute separate demographic analysis using authentic clinical evidence whenever
obtainable, particularly when examining documented disparity risks.

Comparators (Q14–Q15). The comparator should match the device's real workflow and the
realistic alternative care. Comparison is especially weak when outputs are open-ended.
Free-text responses have no single correct answer, physicians agree only moderately when
grading them, and methods for human evaluation of LLMs in health care are not standardized
[34]. AI graders add their own biases, including a tendency to favor outputs from their own
model family, and few medical studies test for these biases [35,36]. A comparator is most
defensible where the output reduces to a discrete clinical decision with a guideline- or
outcome-anchored reference [29,30], where grading uses prespecified critical criteria validated
by multiple physicians [37], or where the task is constrained and scored with a validated
instrument [38]. For open-ended outputs, FDA should accept a comparator only when the
sponsor:

●​ breaks outputs down into discrete, clinically gradable claims, decisions and actions;
●​ anchors the reference in guidelines or outcomes, with prespecified critical-error criteria;
●​ reports physician-physician agreement as the baseline for interpreting device-physician
agreement;
●​ uses blinded, multi-rater panels with adjudication; and
●​ validates any AI grader against physicians and draws it from a different model family
than the device.
Independent third parties (Q16). DFA supports a substantial role for independent evaluators,
including professional societies, academic centers and accredited laboratories, in holding
sequestered datasets and supplying qualified adjudicators [16,39]. Independence must be real:
evaluators should not be selected or paid directly by the sponsor whose device they assess,
financial ties should be disclosed, and third-party assessments should be published or available
on request [39]. Third-party certification should inform, never replace, FDA's own determination.

For high-risk devices, FDA should encourage independent evaluation of the most safety-critical
claims and failure modes. Third-party assessments should be made publicly available or
available upon request to the extent permitted by law and necessary protections for confidential
information. Manufacturers should not be able to select evaluation criteria after observing
results or exclude unfavorable findings without justification.

To preserve competition and access, FDA should permit multiple qualified evaluators rather than
requiring exclusive reliance on a single private organization. The goal should be credible,
independent assessment without creating unnecessary barriers for developers.

Section VI: Postmarket monitoring (Questions 18–24)

Greater reliance on postmarket evidence is acceptable only when the monitoring system
exists at launch, can detect harm before it spreads, and is tied to a prompt response.

Accepting premarket uncertainty (Q18). DFA opposes reduced premarket evidence for
functions that are patient-facing, action-directing, time-critical, autonomous or capable of
irreversible harm. For those devices, the first postmarket signal may be a patient injury. A
promise to build monitoring later should not justify lower premarket evidence. Where FDA does
accept more uncertainty for lower-risk functions, an operational, independently governed
monitoring system should be a necessary but not sufficient condition, and safety-critical
behaviors should be monitored in near real time, not merely periodically [16].

Resources are a concern. MDUFA commitments have not provided dedicated funding for
postmarket evaluation of AI-enabled devices, and in December 2025 the Government
Accountability Office reported that insufficient staffing limits FDA's ability to conduct oversight
[41]. Without dedicated resources, DFA is concerned that the risk of irreversible harm could
exceed what clinicians and patients should reasonably accept. Our 2025 comments
recommended REMS-like risk mitigation for GenAI devices used in vulnerable populations [1].

Even low-consequence, non-directive functions with clinician review may not warrant reduced
premarket testing, because human interaction and workflow effects can produce automation
bias [42] and alarm fatigue [43], degrading physician decisions and putting patients at risk.

Monitoring approaches and cadence (Q19). All three approaches — periodic
re-benchmarking, sample-based clinician review and performance degradation monitoring —
are needed together. Proactive, data-driven tracking of input and output performance, drift and
clinical outcomes should be integrated into the device's quality management system (QMS), at a
fixed minimum interval and whenever triggered by:

●​ any change to the foundation model, prompts, retrieval sources, guardrails, orchestration
logic or user interface [44];
●​ a measurable shift in the user population or setting; or
●​ any safety signal or cluster of user complaints.

Clinician review samples should oversample safety-critical interactions, such as
escalation-relevant conversations, because rare high-consequence failures are
underrepresented in random samples [16]. Our 2025 comments proposed real-time auditing at
defined intervals, within continuous monitoring, against published criteria for transparency, bias
and clinical accountability, including foreseeable off-label uses [1].

Machine-based supervisory agents (Q20). Automated supervisory agents may help monitor
high volumes of interactions, but they cannot be the final safeguard. A supervisor built on similar
architecture may share the blind spots of the device it monitors, especially for the edge cases
most likely to cause harm [16]. Any supervisory agent should itself be validated against clinician
review, drawn from a different model family where feasible, and monitored for drift. Physicians
should review performance periodically throughout deployment, human review should remain
the final layer for any flagged safety-critical event, and a human-controlled mechanism should
allow a device to be suspended immediately.

Shared ecosystem responsibility (Q21). Shared roles must not dilute manufacturer
accountability. The manufacturer holds the data and should bear primary responsibility for
surveillance, investigation and correction. Our 2025 comments called for an organized body of
professional societies, accreditation bodies, clinicians, patients and regulators to help define
monitoring roles [1]. Monitoring must span regions and health systems: peer-reviewed
evaluation of the Epic Sepsis Model found that its performance varied with patient mix across
hospitals [45].

Professional societies, health systems and independent registries can add clinical expertise and
outcome linkage, and can support local postmarket performance monitoring within a lifecycle
approach, as Warraich, Tazbaz and Califf have proposed [46]. The obstacles are significant.
These groups often lack access to manufacturer-held data because of encryption, privacy
restrictions, proprietary systems and nondisclosure agreements, and meaningful review requires
expertise and resources [16]. Participation should not be an unfunded mandate, which would be
an unfair burden and would further disadvantage smaller and rural health systems. Industry
should participate but not control adjudication, publication or access to patient data. DFA also
supports a pooled, cross-manufacturer safety network modeled on the Vaccine Safety Datalink,
because no single sponsor will see population-level signals, this is particularly important
because a single foundation model may underlie many devices.

Authorization should also require a discontinuation plan. Consumer mental health apps have
shut down while users still relied on them, including Woebot in 2025 [47] and Youper in 2026
[48]. Patients who rely on a device daily need advance notice, data access and referral if it is
withdrawn.

Modifications and change control (Q22–Q23). Change control is critical for GenAI devices,
because their most consequential changes can be the hardest to anticipate. Any correction
made to reduce a risk to health must be reported to FDA under 21 CFR Part 806 [49].

A predetermined change control plan (PCCP) is appropriate for anticipated, bounded and
testable changes — such as fine-tuning under a fixed protocol, guardrail refinements or
knowledge-base updates — that are verified against prespecified acceptance criteria with no
degradation in any subgroup, are reversible, and are disclosed to users. A PCCP is not
appropriate for changes whose effects cannot be predicted in advance, most importantly
replacement or updating of the underlying foundation model, which can alter device behavior
broadly and may be controlled by a third party. DFA urges FDA to:

●​ treat a foundation model version change as outside the scope of any PCCP unless the
plan specifies the exact version transition and a full re-benchmarking protocol;
●​ never authorize open-ended future changes to the underlying foundation model through
a PCCP;
●​ when future changes cannot be prespecified, authorize the evaluation protocol and
acceptance criteria in advance, not the change itself; and
●​ require manufacturers to notify users and FDA of every foundation model version
change, with model and version recorded in adverse event reports.

​
DFA proposes three levels of re-evaluation:

Level of When it applies
re-evaluation

Targeted testing Bug fixes that restore the cleared specification; cybersecurity
patches that do not alter function; interface or wording changes that
do not affect how users interpret outputs

Full re-benchmarking Foundation model replaced or updated with intended use
on the existing unchanged; fine-tuning or retraining; changes to system prompts,
benchmark, including guardrails, retrieval sources or safety logic (at minimum, the affected
all safety items functions and all safety items)
 New benchmark New failure modes found postmarket (each confirmed failure
items, plus a new becomes a test item); expanded intended use, users or population,
submission where such as clinician- to patient-facing or adults to children (a major
required change in intended use requires a new 510(k) under 21 CFR
807.81(a)(3)); revised clinical guidelines that make an old answer
wrong; benchmark saturation or contamination; real-world
performance worse than the benchmark predicted

Third-party foundation model changes (Q24). In April 2025, OpenAI released an update to
GPT-4o that made the model markedly more sycophantic, including reinforcing users' delusions,
and rolled it back four days later. OpenAI acknowledged that its deployment evaluations had not
tracked sycophancy before release [50,51]. Any medical device built on that model would have
changed behavior overnight, without notice to the device maker, clinicians or FDA.

FDA should require, as a condition of authorization, enforceable contractual rights to advance
notice of material model changes, version pinning, configuration logs, incident disclosure and
rollback. Changes should be communicated to the health systems and clinicians using the
device. If a manufacturer cannot obtain enough information to evaluate its own dependency, that
is itself an unresolved device risk.

Manufacturers should be required to document and manage their dependence on third-party
foundation models as critical suppliers. Submissions should include an AI bill of materials
identifying the model, version, provider and any provider-applied safety layers; evidence of
contractual version pinning and advance notice of changes; an independent monitoring protocol
that runs a fixed sentinel test set against the deployed model to detect unannounced behavior
changes; a risk analysis covering supplier changes, deprecation, outages and exit; and a
validated contingency plan, including safe degradation and rollback. Where a manufacturer
cannot secure version stability or change notification from its supplier, FDA should treat the
device as higher risk.

Section VII: Other topics (Questions 25–26)

Foundation Model Master Files (Q25). A Foundation Model Device Master File (MAF) is a
useful idea, but a voluntary, confidential file cannot carry the safety weight these models bear.
Because submission would be voluntary, model developers would have limited incentive to
disclose unfavorable safety information. DFA recommends:

●​ Make core disclosure a condition of use, not a voluntary option. A device sponsor
should not be able to rely on a foundation model whose developer will not provide FDA
with training data provenance, known failure modes, subgroup performance, update
commitments and audit log access. If FDA lacks authority to require this of model
 developers directly, it can require it of sponsors as a condition of authorization, and
should tell Congress where authority is lacking.
●​ Keep confidentiality from hiding safety information. Trade secrets can be protected
while still publishing a public summary of each referenced model's known limitations,
subgroup performance and version history.
●​ Do not let a master file reduce device-level testing. Any credit sponsors receive for
referencing an MAF should apply only to model-level characterization, never to
device-level safety benchmarking or clinical confirmation.
●​ Maintain a confidential FDA map linking each authorized device to the model version
beneath it, so the agency can identify every affected product when a model changes or
fails.

Agentic AI (Q26). Agentic GenAI-enabled devices warrant a distinct regulatory approach, not
merely an additional benchmarking element. Unlike advisory systems, agents act: they can
place orders, send patient-facing messages or control other devices, removing the clinician
decision point on which other safeguards depend. Errors can compound across multi-step
tasks. Even the best-performing agent tested failed roughly 30% of realistic, physician-written
EHR tasks in simulation [52], and all leading models tested were susceptible to prompt
injections hidden in clinical content [53].

DFA urges FDA to:

1.​ evaluate the full action chain and treat a system as a device if any action it can take
meets the device definition;
2.​ place autonomous action-taking in the highest risk tier by default;
3.​ require a single accountable manufacturer for the end-to-end system;
4.​ require end-to-end, repeated-run and adversarial testing, followed by staged deployment
through sandbox testing and shadow mode;
5.​ require, not merely evaluate, a bounded action space, human confirmation before
irreversible or high-consequence actions, and a tested stop mechanism, with checkpoint
design validated for genuine human review;
6.​ require complete audit logs, monitoring of approval and override patterns, and near-miss
reporting; and
7.​ treat new tools or action capabilities as requiring new submissions rather than PCCP
updates.

Cross-cutting concerns not addressed in the discussion paper

Competency testing of GenAI devices, like competency testing of clinicians, leaves a large gap
between test performance and clinical practice. Clinicians and patients should be involved
throughout development and evaluation. The ultimate question regulators and sponsors must
answer is whether a device improves the quality of care, or simply adds another layer to an
increasingly fragmented system with no clear accountability.
The paper is silent on several issues that will decide whether clinicians and patients can trust
GenAI-enabled devices. DFA asks FDA to address them in any guidance that follows, and to
coordinate with the Federal Trade Commission (FTC), the states and other authorities so that a
consistent set of rules evolves.

Privacy and data protection

GenAI devices collect unusually intimate data: free-text descriptions of symptoms, mental state,
relationships and behavior. HIPAA protects that data only when it is held by a covered entity or
business associate; health information entered into a consumer app often falls outside HIPAA
altogether [54,56]. The main federal protection for apps outside HIPAA is the FTC's Health
Breach Notification Rule, which requires notice after a breach but does not limit how such data
may be used to train or improve models [55]. Our 2025 comments noted that ownership of
patient data and machine-generated outputs remains unresolved, and that consent must be
obtained and understood [1]. DFA recommends that FDA:

●​ require premarket disclosure of what user data a device collects, where it is stored,
whether it is used to train or fine-tune models, and whether it is shared with the
foundation model developer;
●​ treat data flows to third-party model providers as part of the device's cybersecurity and
risk review;
●​ require consent that is meaningful and capacity-sensitive, with opt-out and correction
rights. Decision-making capacity is condition-specific, and may fluctuate in conditions
such as dementia, delirium and serious mental illness, and should be held to a higher
standard as stakes rise [57]. One-time click-through acceptance is inadequate for these
users. Patient-facing devices intended for or foreseeably used by these populations
should build in ongoing consent, periodic re-confirmation, and a role for a legally
authorized representative or caregiver where appropriate; and
●​ convene, with HHS's Office for Civil Rights and the FTC, the working group on patient
privacy and data ownership that our 2025 comments proposed.

Transparency of the framework, the algorithms and the evidence

DFA appreciates FDA's stated intent to increase transparency in GenAI device regulation. The
discussion paper is a good first step, and we look forward to participating in each phase. The
unique challenges posed by GenAI can be met only with balanced input from all stakeholders.

Physician organizations, regulators and researchers agree on the importance of transparency
but differ on its substance. The American Medical Association (AMA) has called for
transparency about the design, development and deployment of health care AI, mandated by
law where possible [58].However, deviating from this goal, FDA's transparency principles and
model-card recommendations remain voluntary [59], and the proposed Foundation Model MAF
would remain confidential to FDA. Researchers have documented that voluntary disclosure
leaves clinicians without basic performance information [60]. DFA urges FDA to make
standardized, public disclosure — including subgroup and real-world performance — a
requirement for GenAI-enabled devices.

Transparency is a safety control, not a courtesy. Clinicians cannot use a tool responsibly without
knowing how a model was tested, how changes will be handled, and how it performs in the real
world. DFA recommends:

●​ Model cards. FDA's 2025 draft guidance recommends model cards — documentation of
model details, training data, performance, limitations and intended use — and DFA
supports them [59]. But current regulations do not ensure public access to this
information. A 510(k) sponsor may file either a summary or a statement (21 CFR
807.87(h)); a summary need only explain the basis for substantial equivalence (§
807.92), and a statement merely promises to provide information on written request,
excluding trade secret and confidential commercial information (§ 807.93). A
recommendation in draft guidance does not change this. Model cards should be required
and made public.
●​ Labeling. Labeling must be audience-specific and aligned with FDA's Good Machine
Learning Practice principles. Labeling statements should be supported by evidence in
marketing submissions; how outputs are meant to be used should be clearly stated and
embedded in the device interface; and known limitations should be included.
●​ Performance data. FDA has not required sponsors to publish summaries of
benchmarking or clinical confirmation tests. The discussion paper treats this as part of
internal review, with possible voluntary sharing through a Foundation Model MAF. A
mandate is needed so that clinicians and patients understand the strengths and
weaknesses of the evidence supporting a device.
●​ Version disclosure. Require notice to deploying institutions and users when an update
or foundation model change meaningfully alters behavior.
●​ Real-world performance. Publish periodic public reports of postmarket findings,
including adverse events and corrective actions.
●​ The framework itself. Publish how the risk framework is applied to authorized devices,
so patients and clinicians can see why a device was placed where it was.

Bias in training and evaluation datasets

Foundation models learn from data that reflect existing inequities, and bias in training data is a
documented source of harm, not a theoretical one. Examples are numerous and well known to
clinicians. A widely used commercial algorithm cut the share of Black patients identified for extra
care roughly in half because it learned from historical spending patterns [61]. Imaging AI
systematically underdiagnosed women and Black, Hispanic and Medicaid patients [62]. LLMs
show the same pattern: GPT-4 was less likely to recommend advanced imaging for Black
patients and cardiac stress testing for women [63], and across nine LLMs and 1,000 emergency
cases, identical presentations received different recommendations by race, income, housing
status and sexual orientation [64]. Clinicians cannot reliably detect such bias case by case. A
device built on these models inherits these patterns, and the sponsor may have no visibility into
the training data.
DFA reiterates and expands on its 2025 recommendations:

●​ A validated, systematic method for evaluating demographic bias should be part of the
total product life cycle approach, consistent with FDA's 2025 draft guidance on
AI-enabled device software functions [59].
●​ FDA should require prespecified, subgroup-stratified bias testing using adversarial
methods.
●​ Subgroup performance must be reported in a form that lets clinicians judge whether a
device works for their patients. For each prespecified subgroup (at minimum age,
gender, race and ethnicity, primary language, and relevant clinical characteristics such
as comorbidity or disease severity), sponsors should report the number of patients or
cases tested, the performance estimate and its 95% confidence interval. Without
denominators and confidence intervals, a reported "no difference" between groups is
uninterpretable: a subgroup of 40 patients will produce intervals consistent with both
comparable and substantially worse performance. Absence of evidence of a difference is
not evidence of equivalence [65]. FDA should therefore require sponsors to classify each
subgroup as (1) demonstrated comparable performance, meeting a prespecified
equivalence or non-inferiority margin; (2) demonstrated inferior performance; or (3)
insufficient evidence, where the interval is too wide to tell. Subgroups with insufficient
evidence should be disclosed as such in labeling and the public summary, not reported
as showing no disparity, and should be priorities for postmarket evaluation. This reflects
established reporting standards for clinical prediction models and LLMs (TRIPOD+AI
and TRIPOD-LLM) [66,67] and FDA's own guidance on demographic subgroup reporting
in device studies [68], and addresses the near-total absence of such reporting in current
FDA summaries [60].
●​ Testing should cover race and ethnicity, age bands, pregnancy, disability, language,
dialect and health literacy.
●​ Patients should be informed of known limitations in subgroup performance before use.

Liability and accountability

The competency analogy that a GenAI device can be evaluated the way we assess a clinician,
assumes an accountability system that does not yet exist for devices. Licensure rests not only
on examinations but on years of closely supervised practice, in which clinical educators test for
failure points in the reasoning and decisions of residents every day. Building a digital analog
would be complex, and so far the gap between exam performance and clinical performance has
not been closed [18].

When a device operating within its labeling contributes to harm, responsibility should not fall on
the physician simply because a human was nominally in the loop. The AMA has advocated
limiting physician liability for AI-enabled tools and placing liability on those best positioned to
know and mitigate the risk, including developers of autonomous systems [58]. FDA's labeling
and authorization decisions do not determine tort liability, but courts look to them, which makes
this an area of serious concern for physicians.
The regulatory pathway also matters. Under Riegel v. Medtronic, most state tort claims involving
devices approved through premarket approval are preempted, while claims involving
510(k)-cleared devices generally are not [69,70]. Our recommendation that high-risk GenAI
devices go through premarket approval could therefore narrow patients' recourse against
manufacturers and leave clinicians as the remaining defendants. This tension should be
addressed, not ignored. DFA asks FDA to:

●​ state clearly in labeling and authorization decisions what level of clinician oversight the
device's evidence actually supports, so that "human in the loop" is not used to shift risk
onto clinicians;
●​ keep manufacturers responsible for the third-party model components they incorporate;
and
●​ raise the liability gap, including the effect of preemption, with HHS, state authorities and
Congress.

Adequacy of regulation and industry influence

DFA is concerned that the framework is being shaped by those it would regulate. The paper's
most consequential proposals — sponsor-developed benchmarks, voluntary master files,
third-party certification and acceptance of greater premarket uncertainty — each shift judgment
from FDA toward industry. Market pressure is a real factor: publicly traded firms made about half
of AI devices but accounted for more than 90% of recalls, and their recalled devices more often
lacked clinical validation [3]. DFA recommends that FDA:

●​ ensure that clinicians, patients, independent researchers and public health experts are
represented at least equally with industry in workshops, advisory committees, standards
bodies and any consortium that develops benchmarks;
●​ publicly disclose meetings and substantive communications with industry on this
framework, and the conflicts of interest of advisors and third-party evaluators;
●​ fund independent evaluation infrastructure, such as sequestered datasets and
adjudication, preferably through congressional appropriations; any user-fee funding
should be firewalled so that it does not depend on payment from the sponsor whose
device is assessed and gives industry no influence over evaluation;
●​ publish a summary of comments received on this docket and how they were weighed;
and
●​ report to Congress where existing authority is inadequate, including over foundation
model developers, non-device clinical decision support and consumer-facing chatbots.

As our 2025 comments noted, rapid model updates can outpace FDA's review capacity [1]. FDA
will need dedicated staff, technical capacity and sustained funding to oversee these devices,
independent of the industry it reviews.
GenAI-enabled devices may help patients who cannot reach care today, and DFA wants them to
succeed, but they will succeed only if patients and physicians can trust them. That trust
depends on evidence of benefit proportionate to risk, independent and enforceable postmarket
oversight, mandatory transparency, explicit attention to bias and privacy, fair allocation of
accountability, and a regulatory process that is not shaped disproportionately by the industry it
regulates. DFA would welcome the opportunity to work with CDRH, professional societies and
patient groups as this framework develops.

Please contact our Task Force AI Committee leader, Dr. Raashid Khan, at raashidk@gmail.com,
with any questions.

Respectfully submitted,

FDA Task Force, Doctors for America

References

1.​ Doctors for America FDA Task Force. Public comment on the Digital Health Advisory
Committee Meeting on Generative Artificial Intelligence-Enabled Digital Mental Health
Medical Devices. November 2025.
2.​ US Food and Drug Administration. Considerations for the Regulation of Generative
AI-Enabled Medical Devices: Discussion Paper and Request for Feedback. Docket No.
FDA-2026-N-7874. 2026.
3.​ Lee B, Kramer P, Sandri S, et al. Early recalls and clinical validation gaps in artificial
intelligence–enabled medical devices. JAMA Health Forum. 2025;6(8):e253172.
doi:10.1001/jamahealthforum.2025.3172
4.​ US Food and Drug Administration. Executive Summary for the Digital Health Advisory
Committee Meeting: Total Product Lifecycle Considerations for Generative AI-Enabled
Devices. November 2024.
5.​ Vasan N; Brainstorm: The Stanford Lab for Mental Health Innovation. Feedback on
CDRH's discussion paper on generative AI-enabled medical devices. Docket No.
FDA-2026-N-7874. August 19, 2026.
6.​ Vedder. WHOOP, blood pressure and FDA's new wellness framework: what changed –
and what companies can do now. JD Supra. September 2026.
7.​ Strickland E. How IBM Watson overpromised and underdelivered on AI health care.
IEEE Spectrum. April 2019.
8.​ Kyomen HH; Group for the Advancement of Psychiatry Committee on Aging. Am J
Geriatr Psychiatry. 2026;34:674–692. doi:10.1016/j.jagp.2025.12.012
9.​ Common Sense Media Youth AI Safety Institute; Stanford Brainstorm Lab. AI mental
health apps risk assessment. May 5, 2026.
10.​Dratsch T, Chen X, Rezazade Mehrizi M, et al. Automation bias in mammography: the
impact of artificial intelligence BI-RADS suggestions on reader performance. Radiology.
2023;307(4):e222176. doi:10.1148/radiol.222176
11.​Jabbour S, Fouhey D, Shepard S, et al. Measuring the impact of AI in the diagnosis of
hospitalized patients: a randomized clinical vignette survey study. JAMA.
2023;330(23):2275–2284. doi:10.1001/jama.2023.22295
12.​Bean AM, Payne RE, Parsons G, et al. Reliability of LLMs as medical assistants for the
general public: a randomized preregistered study. Nat Med. 2026;32(2):609–615.
doi:10.1038/s41591-025-04074-y. Erratum in: Nat Med. 2026;32(5):1937.
13.​Heinz MV, Mackin DM, Trudeau BM, et al. Randomized trial of a generative AI chatbot
for mental health treatment. NEJM AI. 2025;2(4). doi:10.1056/AIoa2400802
14.​Common Sense Media Youth AI Safety Institute. AI chatbots for mental health support:
risk assessment. November 14, 2025.
15.​ChatGPT Health triage advice falls short in key cases. Nat Med. 2026;32:1600–1601.
doi:10.1038/s41591-026-04427-1
16.​American College of Allergy, Asthma and Immunology. Comment on Docket No.
FDA-2026-N-7874. September 25, 2026.
17.​Bedi S, Liu Y, Orr-Ewing L, et al. Testing and evaluation of health care applications of
large language models: a systematic review. JAMA. 2025;333(4):319–328.
doi:10.1001/jama.2024.21700
18.​Hager P, Jungmann F, Holland R, et al. Evaluation and mitigation of the limitations of
large language models in clinical decision-making. Nat Med. 2024;30(9):2613–2622.
doi:10.1038/s41591-024-03097-1
19.​Goh E, Gallo R, Hom J, et al. Large language model influence on diagnostic reasoning:
a randomized clinical trial. JAMA Netw Open. 2024;7(10):e2440969.
doi:10.1001/jamanetworkopen.2024.40969
20.​Shikino K, Shimizu T, Otsuka Y, et al. Evaluation of ChatGPT-generated differential
diagnosis for common diseases with atypical presentation: descriptive research. JMIR
Med Educ. 2024;10:e58758. doi:10.2196/58758
21.​Liu NF, Lin K, Hewitt J, et al. Lost in the middle: how language models use long contexts.
Trans Assoc Comput Linguist. 2024;12:157–173.
22.​Adams L, Busch F, et al. LongHealth: a question answering benchmark with long clinical
documents. arXiv:2401.14490. January 2024.
23.​Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and
quality of care for older patients with multiple comorbid diseases: implications for pay for
performance. JAMA. 2005;294(6):716–724. doi:10.1001/jama.294.6.716
24.​Lee Y, Hong S, Yon DK. Artificial intelligence in pediatric healthcare: bridging potential,
clinical practice, and ethical considerations. Clin Exp Pediatr. 2025;68(9):652–655.
doi:10.3345/cep.2025.01844
25.​Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research
outcome. Cochrane Database Syst Rev. 2017;2:MR000033.
doi:10.1002/14651858.MR000033.pub3
26.​Bedi S, Jiang Y, Chung P, Koyejo S, Shah N. Fidelity of medical reasoning in large
language models. JAMA Netw Open. 2025;8(8):e2526021.
doi:10.1001/jamanetworkopen.2025.26021
27.​Wu E, Wu K, Daneshjou R, et al. How medical AI devices are evaluated: limitations and
recommendations from an analysis of FDA approvals. Nat Med. 2021;27:582–584.
doi:10.1038/s41591-021-01312-x
28.​International Medical Device Regulators Forum. Software as a Medical Device: Possible
Framework for Risk Categorization and Corresponding Considerations. IMDRF/SaMD
WG/N12FINAL:2014.
29.​Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous
AI-based diagnostic system for detection of diabetic retinopathy in primary care offices.
NPJ Digit Med. 2018;1:39.
30.​Ramaswamy A, Tyagi A, Hugo H, et al. ChatGPT Health performance in a structured test
of triage recommendations. Nat Med. 2026;32:1671–1675.
doi:10.1038/s41591-026-04297-7
31.​Wong A, Otles E, Donnelly JP, et al. External validation of a widely implemented
proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med.
2021;181(8):1065–1070. doi:10.1001/jamainternmed.2021.2626
32.​Shumailov I, Shumaylov Z, Zhao Y, et al. AI models collapse when trained on recursively
generated data. Nature. 2024;631:755–759. doi:10.1038/s41586-024-07566-y
33.​Chen RJ, Lu MY, Chen TY, et al. Synthetic data in machine learning for medicine and
healthcare. Nat Biomed Eng. 2021;5:493–497. doi:10.1038/s41551-021-00751-8
34.​Tam TYC, Sivarajkumar S, Kapoor S, et al. A framework for human evaluation of large
language models in healthcare derived from literature review. NPJ Digit Med.
2024;7:258.
35.​Li C. A scoping review of LLM-as-a-judge in healthcare and the MedJUDGE framework.
arXiv preprint. 2026.
36.​Pombal J, Rei R, et al. Self-preference bias in rubric-based evaluation of large language
models. arXiv preprint. 2026.
37.​Arora RK, et al. HealthBench: evaluating large language models towards improved
human health. OpenAI. 2025.
38.​Croxford E, Gao Y, First E, et al. Evaluating clinical AI summaries with large language
models as judges. NPJ Digit Med. 2025.
39.​Congress of Neurological Surgeons. Comment on Docket No. FDA-2026-N-7874. 2026.
40.​Daneshjou R, Vodrahalli K, Novoa RA, et al. Disparities in dermatology AI performance
on a diverse, curated clinical image set. Sci Adv. 2022;8(31):eabq6147.
41.​US Government Accountability Office. Medical Device Recalls: HHS and FDA Should
Address Limitations in Oversight of Recall Process. GAO-26-107619. December 2025.
42.​Astobiza AM, Alonso M, Ortega Lozano R. Trust and AI in healthcare: a systematic
review. Monash Bioeth Rev. 2025. doi:10.1007/s40592-025-00272-z
43.​Michels EAM, Gilbert S, Koval I, et al. Alarm fatigue in healthcare: a scoping review of
definitions, influencing factors, and mitigation strategies. BMC Nurs. 2025;24:664.
doi:10.1186/s12912-025-03369-2
44.​Crowell & Moring LLP. FDA proposes GenAI medical device regulatory framework: what
manufacturers need to know. 2026.
45.​Lyons PG, Hofford MR, Yu SC, et al. Factors associated with variability in the
performance of a proprietary sepsis prediction model across 9 networked hospitals in the
US. JAMA Intern Med. 2023;183(6):611–612. doi:10.1001/jamainternmed.2022.7182
46.​Warraich HJ, Tazbaz T, Califf RM. FDA perspective on the regulation of artificial
intelligence in health care and biomedicine. JAMA. 2025;333(3):241–247.
doi:10.1001/jama.2024.21451
47.​Drut M. The great AI divide: are mental health apps leaving consumers behind?
February 2025.
48.​Pennock S, Callans M. Youper review (2026): the AI health assistant, shutting down in
2026. Psychology.com. September 2026.
49.​Reports of corrections and removals. 21 CFR Part 806.
50.​OpenAI. Expanding on what we missed with sycophancy. May 2025.
51.​Georgetown Law Institute for Technology Law & Policy. Tech brief: AI sycophancy &
OpenAI. 2025.
52.​Jiang Y, Black K, Geng G, et al. MedAgentBench: a virtual EHR environment to
benchmark medical LLM agents. NEJM AI. 2025;2(9). doi:10.1056/AIdbp2500144
53.​Clusmann J, Ferber D, Wiest IC, et al. Prompt injection attacks on vision language
models in oncology. Nat Commun. 2025;16:1239. doi:10.1038/s41467-024-55631-x
54.​US Department of Health and Human Services, Office for Civil Rights. Health app use
scenarios & HIPAA. 2016.
55.​Federal Trade Commission. Health Breach Notification Rule. 16 CFR Part 318 (as
amended 2024).
56.​Shachar C, Cadario R, Cohen IG, et al. HIPAA is a misunderstood and inadequate tool
for protecting medical data. Nat Med. 2023;29:1900–1902.
doi:10.1038/s41591-023-02355-y
57.​Appelbaum PS. Assessment of patients' competence to consent to treatment. N Engl J
Med. 2007;357:1834–1840. doi:10.1056/NEJMcp074045
58.​American Medical Association. Principles for Augmented Intelligence Development,
Deployment, and Use. November 2024.
59.​US Food and Drug Administration. Artificial Intelligence-Enabled Device Software
Functions: Lifecycle Management and Marketing Submission Recommendations. Draft
guidance. January 2025.
60.​Muralidharan V, Adewale BA, Huang CJ, et al. A scoping review of reporting gaps in
FDA-approved AI medical devices. NPJ Digit Med. 2024;7:273.
doi:10.1038/s41746-024-01270-x
61.​Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm
used to manage the health of populations. Science. 2019;366(6464):447–453.
doi:10.1126/science.aax2342
62.​Seyyed-Kalantari L, Zhang H, McDermott MBA, Chen IY, Ghassemi M. Underdiagnosis
bias of artificial intelligence algorithms applied to chest radiographs in under-served
patient populations. Nat Med. 2021;27(12):2176–2182.
doi:10.1038/s41591-021-01595-0
63.​Zack T, Lehman E, Suzgun M, et al. Assessing the potential of GPT-4 to perpetuate
racial and gender biases in health care: a model evaluation study. Lancet Digit Health.
2024;6(1):e12–e22. doi:10.1016/S2589-7500(23)00225-X
64.​Omar M, Soffer S, Agbareia R, et al. Sociodemographic biases in medical decision
making by large language models. Nat Med. 2025;31(6):1873–1881.
doi:10.1038/s41591-025-03626-6
65.​Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ.
1995;311:485.
66.​Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for
reporting clinical prediction models that use regression or machine learning methods.
BMJ. 2024;385:e078378.
67.​Gallifant J, Afshar M, Ameen S, et al. The TRIPOD-LLM reporting guideline for studies
using large language models. Nat Med. 2025.
68.​US Food and Drug Administration. Evaluation and Reporting of Age-, Race-, and
Ethnicity-Specific Data in Medical Device Clinical Studies. Guidance. September 2017.
69.​Riegel v. Medtronic, Inc., 552 U.S. 312 (2008).
70.​Medtronic, Inc. v. Lohr, 518 U.S. 470 (1996).