Yale Collaboration for Regulatory Rigor Integrity and Transparency (CRRIT)

Please see attached document.

U.S. Department of Health and Human Services Food and Drug Administration
5630 Fishers Lane, Rm. 1061, Rockville, MD 20852
October 19, 2026

[Submitted Online]

RE: FDA-2026-N-7874 Considerations for the Regulation of Generative AI-Enabled Medical
Devices: Discussion Paper and Request for Feedback

To Whom it May Concern,

Thank you for the opportunity to respond to the U.S. Food and Drug Administration (FDA) request
for public comment on the Center for Devices and Radiological Health (CDRH) discussion paper,
Considerations for the Regulation of Generative AI enabled Medical Devices. We are members of
the Yale Collaboration for Regulatory, Rigor, Integrity, and Transparency (CRRIT), an
interdisciplinary initiative aligning research on medical product evaluation, approval, and
coverage with the goal of improving patient health and healthcare. We appreciate CDRH's efforts
to engage stakeholders early and openly of substantial and growing public health importance.
Our comments center on the idea that current considerations appear to prioritize clinical validation,
performing as well as a clinician on a defined test, as opposed to clinical utility, ensuring that use
of the technology meaningfully improves clinical outcomes for patients. Specifically for patientfacing conversational generative AI (GenAI), which is of primary concern for this letter, risk
depends on the user and the conversation, not just the product. The same system can answer a
general question about a condition in one exchange and triage a potential emergency in the next,
depending entirely on what a patient writes. Therefore, risk is dynamic within a single encounter:
a conversation that begins as a request for general information can become a triage decision as
symptoms are shared, so the risk of an interaction is not fixed when it starts. Risk also depends on
the environment in which a tool operates and the data it has access to. For example, a tool
integrated with a health system’s electronic health record can draw on a patient’s history,
medications, and comorbidities, whereas a standalone consumer tool must rely solely on what the
patient reports. Therefore, for these devices the unit of validation should shift from the isolated
output to the clinical interaction, given this dynamic risk.
Second, a competency-based framework for GenAI-enabled devices should preserve three distinct
evidentiary questions, each requiring its own evidence: analytical validity (does the system
produce reliable, reproducible, safety-critical behavior across realistic conversational variation?);
clinical validity (are those behaviors clinically appropriate across representative scenarios and
populations?); and clinical utility (does patient interaction with the system lead to better real-world
decisions or outcomes?). These terms follow the definitions in the FDA-NIH BEST Resource,
which treats clinical utility, a net improvement in health outcomes, as distinct from analytical and
clinical validity.1 In addition, FDA’s guidance on the clinical evaluation of software as a medical
device, adopted from the International Medical Device Regulators Forum (IMDRF), uses related
terms (valid clinical association, analytical validation, and clinical validation) but folds a tool’s
effect on patients into clinical validation2; we recommend treating clinical utility as its own
question. Throughout, we use “patient-facing” to describe tools whose intended user is a patient
or member of the public, as distinct from tools intended for healthcare professionals (HCPs),
consistent with the terminology in Section II of the CDRH discussion paper3 and the user-type
distinctions in recent ESMO guidance on the use of large language models in clinical practice
(ELCAP).4 Below, we address several of the discussion paper’s numbered questions, drawing
where relevant on an ongoing empirical research project described in the following paragraph.
Our comments are partially informed by ongoing research that has not yet been published. We are
conducting a cross-sectional study of publicly accessible, patient-facing GenAI health and medical
chatbots. Using a structured search strategy across two search engines and four search terms, we
identified 101 distinct publicly accessible, GenAI health and medical chatbots. From this set, we
selected the 23 most common tools for in-depth characterization using a standardized extraction
framework informed by FDA AI/SaMD guidance. We extracted information on each tool’s
intended uses, clinical functions/claims, target users, accessibility, and– most relevant to this
CDRH discussion paper– the validation evidence and safety governance practices developers
disclosed (or failed to disclose) to the public. We explicitly differentiated between analytical
validity, clinical validity, and clinical utility. We applied a product-relevance rule: evidence
counted toward a tool only if it evaluated the actual consumer-facing product, not the underlying
foundation model, an earlier research prototype, or a related clinician-facing product from the same
developer. Of the 23 tools, none held any form of FDA marketing authorization as a GenAIenabled device software function (the discussion paper’s term) for the characterized function.
Most, in other words, are likely not marketed as regulated devices at all, a point we return to in
Section I. We offer the figures below as preliminary descriptive findings from the sample of 23
tools, not as a generalizable estimate of the full market, and reference them where relevant to
indicate how our observations of the current market inform, and in our view reinforce the urgency
of, the questions CDRH has posed.
Our overarching recommendation is straightforward. When a GenAI-enabled product delivers
patient-specific clinical information directly to patients, without a healthcare professional
positioned to review, interpret, or correct its output, FDA should expect systematic premarket
evaluation of that product to be submitted as part of the application for marketing authorization,
with each of the three types of evidence (analytical validity, clinical validity, and clinical utility)
generated, reported, and reviewed in its own right. In HCP-facing settings, the clinician serves as
a check on erroneous output, filtering what a tool produces against their own clinical knowledge.5
In direct-to-patient settings that check is absent, and the evidence must carry the weight the
clinician otherwise would. By systematic, we mean evaluation that is prespecified, conducted or
verified independently of the developer, publicly reported in enough detail to be interpreted, and
repeated when the product changes.

The depth of each type of evidence may reasonably scale with risk, but none should be absent.
This principle holds even within proposals designed to give clinical AI more regulatory flexibility.
Former FDA Commissioner Scott Gottlieb recently proposed a pathway, modeled on the oversight
of laboratory-developed tests, under which FDA would certify health systems to develop and
deploy their own clinical AI; even that proposal would sequence evidence by risk rather than waive
it, and it expressly excludes patient-facing diagnostic tools, which would remain subject to
conventional device review.6 Risk can also be managed at the point of access. A tool could screen
users before allowing them to proceed, and a health system that integrates a tool with its electronic
health record could limit access automatically to patients whose medications, comorbidities, and
history fall within the population for which the tool was validated. Where such gating is used, it
should itself be validated and disclosed. Gating cannot be the only safeguard, however: users who
are screened out may turn to another tool or have someone else use it on their behalf, so the device
itself must also communicate uncertainty in a way that discourages patients from taking its
recommendations at face value (see our response to Question 3).
We recognize that FDA’s authority attaches to a product’s intended use, and Section I describes
how that boundary can be drawn to reflect what these tools actually do. Where products fall outside
FDA’s current authority, we would encourage the agency to say so publicly, so that the gap is
visible to patients, clinicians, and policymakers. The sections that follow apply this principle to
specific questions, but we urge CDRH to state the principle itself explicitly rather than leave it to
be inferred from guidance on individual issues.

I. Device Scope and Intended Use: When Function Is Set at Runtime

Many patient-facing GenAI tools, including the health and medical chatbots now widely available
to the public, are not currently marketed or regulated as medical devices; their developers typically
describe them as informational or general wellness products. Many are also freestanding—they
are not integrated with an electronic health record or other patient data, so their recommendations
are made without full knowledge of the patient’s history. In our view, when these tools respond to
a patient’s description of symptoms with an assessment, a triage recommendation, or advice about
whether and where to seek care, they are performing device functions, and they should be regulated
as medical devices. Use is already widespread: in a nationally representative 2025 survey, 19.2%
of US adolescents and young adults aged 12 to 21 years had used AI chatbots for mental health
advice, and most had not told anyone that they did so.7 We therefore believe they fall squarely
within the scope of this discussion paper. The difficulty is that, under the current approach to
intended use, many of these tools may never enter the framework at all.
Whether software is a medical device depends on its intended use (FD&C Act § 201(h)),8 and
intended use is established by the objective intent of the manufacturer, as shown by labeling
claims, advertising, and the circumstances surrounding distribution (21 CFR 801.4).9 That
approach works well for products whose function is fixed when they are manufactured. For
conversational GenAI, function is substantially set by the user at runtime. A tool labeled as a
general health information resource will, if a patient describes chest pain and shortness of breath,
perform something functionally indistinguishable from triage.

The consequence is that the same clinical risk can fall on either side of the device line depending
only on labeling. Two tools that respond identically to a patient describing stroke symptoms may
be regulated very differently if one describes itself as a symptom checker and the other states that
it is “for informational purposes only” or positions itself within FDA’s General Wellness
guidance.10 This structure rewards developers for disclaiming medical use while providing medical
function. Data from our ongoing work illustrate the gap between claims and capabilities: 18 of 23
tools in our sample stated that they do not provide medical advice or are not intended to diagnose
or treat, yet all of those same tools offered structured symptom intake, triage or care-level
recommendations, or the ability to upload and interpret personal medical records.
The clinical decision support (CDS) exclusion does not close this gap for patient-facing tools.
Section 520(o)(1)(E) of the FD&C Act,11 as interpreted in FDA’s January 2026 CDS guidance,12
excludes software from the device definition only when all four statutory criteria are met, including
that the software is intended to support or provide recommendations to an HCP (criterion 3) and
to enable that professional to independently review the basis for the recommendation (criterion 4).
A tool whose sole user is a patient cannot meet criterion 3. For HCP-facing GenAI, we would add
that a fluent, plausible rationale generated alongside a recommendation should not by itself satisfy
criterion 4. A GenAI-generated explanation can read as a basis without accurately reflecting how
the output was produced, in two ways. First, the rationale is itself generated text, written alongside
the recommendation: it can offer a plausible justification for a wrong recommendation, or different
justifications for the same one. Second, the sources or guidelines it cites may be misattributed, or
may not support the statements attached to them. Criterion 4 should be satisfied only where the
stated basis is dependable to the sources the system relied upon and can be verified by the
reviewing clinician, for example through citations to retrievable sources, ideally the peer-reviewed
literature or established clinical practice guidelines.
We therefore ask CDRH to clarify two points. First, whether health-specific design features—such
as structured symptom intake, triage or urgency outputs, or the ability to upload medical records
or test results—will be treated as objective evidence of intended use under 21 CFR 801.4,9
regardless of disclaimers elsewhere in the product’s labeling. Second, whether CDRH intends to
classify a GenAI-enabled function by the highest-risk function a tool can reach in ordinary use,
rather than by the lowest-risk function its labeling describes. We believe both clarifications are
necessary for the risk framework in the discussion paper to apply to the products patients are
actually using.
II. A Risk Framework for Patient-Facing Conversation (Questions 1, 3, 5, and 6)
The CDRH discussion paper asks whether and when a GenAI-enabled function that delivers
clinical information directly to patients, rather than to HCPs, presents different or higher risk, and
what safeguards might mitigate that risk without underestimating patient capability. We agree with
the paper’s premise that patients, lacking clinical training, may be less able than HCPs to recognize
an incorrect or subtly misaligned output and to avoid relying on it. We would add that this
asymmetry is not well captured by evaluation approaches that test a system’s response to a single,
well-formed prompt in isolation. Real conversations drift—a lay user may express symptoms
ambiguously, minimize a serious complaint, change the subject, or become anxious or persistent,
and a system’s safety-critical behavior can degrade over the course of an exchange even when its
first response is appropriate.
In response to Question 1, we suggest that the two-axis risk framework, which parallels the IMDRF
SaMD framework’s pairing of the significance of the information provided with the state of the
healthcare situation,13 omits four modifiers that matter particularly for GenAI. The first is error
detectability: whether the intended user can reasonably recognize an incorrect output before acting
on it. A subtly miscalibrated reassurance delivered to a patient is far less detectable than the same
error delivered to a clinician. The second is input control. Unlike software that receives structured,
validated inputs, a conversational system depends on whatever the user chooses to disclose, in
whatever order and detail, so the information on which it acts may be incomplete in ways neither
party recognizes. Integration can partly restore input control: a tool connected to the electronic
health record can check what a patient reports against documented history, and its use can be
limited to a narrower scope. The third is correlated error at scale. A systematic failure in a widely
used GenAI tool may be replicated for every user who presents similarly, potentially at very large
volume. The fourth is deployment environment. The same underlying model may present different
risks when it is integrated in a health system. With access to the medical record and clinicians who
can see its outputs (rather than in a public-facing tool, like those in our sample that relies entirely
on what the patient chooses to report) the populations served may also differ in clinical complexity.
Performance demonstrated in one environment should not be assumed to transfer to the other.
Moreover, sponsors should state publicly the environment in which a device was validated, and
new evidence should be required before a device validated in one environment is deployed in
another. We would encourage CDRH to treat each of these as a factor that can raise a function’s
risk category.
In response to Question 3, we urge CDRH to clearly state that disclaimers are not risk mitigation.
A statement that a tool “is not a substitute for professional medical advice” does not change what
a patient does after being told their symptoms are probably not serious, typically presented once
and often outside the conversation itself, to offset a specific, confident, and personalized response
within it. Mitigations should be judged by their demonstrated effect on user behavior and
outcomes, consistent with FDA’s human factors principles,14 not by their presence. The form of
each response matters as much as any disclaimer. Clinicians routinely communicate uncertainty
(“the most likely diagnosis is X, but we will reassess if your symptoms change”), and patientfacing devices should do the same: their language should be concordant to the strength of the
evidence behind a recommendation rather than uniformly confident and declarative, and it should
tell users when to seek care if their symptoms change. CDRH could also consider whether devices
should disclose a level of confidence, provided that its accuracy is itself validated.

In response to Question 5, Section IV of the CDRH discussion paper recognizes that a device may
begin a conversation performing a non-directive informational function and migrate, over the
course of an exchange, toward an action-directing one. We would encourage CDRH to follow that
observation to its conclusion: for a conversational device, intended use should be understood as
the set of functions a conversation can reach, not the function in which it typically begins.
Analytical validity, clinical validity, and clinical utility should then be established through separate
evidentiary evaluations across that reachable set, rather than inferred from one another.
Our ongoing project suggests how far current practice is from this standard. Of the 23 patientfacing tools we reviewed, 14 (61%) had no validation evidence of the consumer-facing product of
any kind (published, preprinted, or even self-reported). Among the 9 tools with such evidence,
none addressed analytical validity, 8 addressed clinical validity, and 1 addressed clinical utility,
and that tool’s evidence measured users’ stated intent to seek care rather than what they actually
did. Where clinical validity evidence was disclosed at all, it was overwhelmingly output-centric,
characterizing the correctness of a single response to a single input; it rarely characterized behavior
across realistic, multi-turn conversations with lay users. A system that is analytically reliable and
clinically accurate on single-turn questions has not thereby demonstrated that patients who interact
with it make better decisions or seek care when they should.
In response to Question 6, we recommend that sponsors of triage or escalation functions report
under-escalation and over-escalation separately, each against a pre-specified acceptance threshold,
rather than as a single accuracy figure. The two errors carry different harms. Under-escalation risks
delayed care for a time-sensitive condition; over-escalation risks unnecessary emergency visits
and other health care use, wastes resources, and erodes trust in future warnings. An aggregate
figure can hide an unacceptable rate of the former behind a favorable rate of the latter. Emergency
care already offers a model: the American College of Surgeons sets separate benchmarks for undertriage and over-triage of injured patients, deliberately tolerating more over-triage in order to keep
under-triage low (commonly cited as under-triage below 5% and over-triage of 25–35%).15 The
February 2026 independent evaluation by Ramaswamy et al. in Nature Medicine,16 which found
that one widely used GenAI health assistant under-triaged 52% of physician-adjudicated true
emergencies, shows why this matters: a failure of that magnitude on the safety-critical side is
immediately visible under separate reporting but can be diluted in a blended metric. We would
encourage CDRH to apply this structure to patient-facing scenarios such as the one described in
Appendix A (S.1) of the discussion paper.
The same asymmetry bears on human oversight. For functions that can reach high-risk, timesensitive scenarios, we recommend that CDRH expect three safeguards: a qualified clinician
available on the back end to review or take over escalated conversations; a prespecified mechanism
that suspends autonomous advice and directs the user to care when a conversation reaches high
risk (a so-called kill switch); and transparency about the sources the device draws on. Human
monitoring, however, is far better at catching one error than the other. A reviewer can readily audit
the cases a device flags for escalation and is therefore well positioned to detect over-escalation.
Under-escalation is much harder to see: a falsely reassured patient generates no flag, may never
return to the tool, and any resulting harm is likely to surface elsewhere, if at all. A suspension
mechanism is only as useful as the signal that triggers it. We therefore recommend that CDRH
place the primary burden for controlling under-escalation on testing rather than surveillance.
Sponsors should demonstrate under-escalation performance against prespecified thresholds before
authorization, and should repeat that testing, using safety-critical scenarios held by an independent
party, whenever the model, system prompts, or retrieval sources change and at any periodic
reassessment or recertification.

III. Benchmarks Define the “Winner”: Construct Validity, Statistical Standards, and
Independence (Questions 9, 10, 12, and 14)

The CDRH discussion paper asks what evidence should support the construct validity of a
benchmark used in device evaluation and recognizes that publicly available benchmarking assets
may be subject to data contamination, saturation, and limited real-world representativeness. We
believe this question deserves particular emphasis, because the choice of benchmark does not
merely measure competing systems—it defines what “competent” means. Published comparisons
of clinical GenAI systems have reached differing conclusions about which tool performs better,
and in our view such disagreements often reflect differences in what the underlying benchmarks
were designed to measure (accuracy against a reference answer, adherence to a guideline, or
something closer to real-world usefulness) rather than a stable difference in system quality.
Multidimensional efforts such as MedHELM17 and CSEDB18 make the same point from the other
direction: when systems are assessed across many clinical task categories, or along separate safety
and effectiveness dimensions, relative performance shifts with the dimension measured. A
benchmark designed around one construct will reliably favor systems optimized for that construct;
changing the construct can change the ranking entirely, in much the same way that a test of treeclimbing and a test of swimming will not produce the same winner between a fish and a squirrel.

An example illustrates this concern. The Ramaswamy et al. evaluation16 described above was
independent, peer-reviewed, received no developer funding, and benchmarked the assistant against
60 gold-standard emergency-case vignettes adjudicated by physician consensus. Within weeks, a
competing developer publicly reported that its own system had correctly identified 100% of the
same 60 cases—a result disclosed only in a company press release, generated by the developer’s
own internal testing, and not independently verified. This example illustrates how two evaluations
against the identical benchmark can be presented to the public as though they answered the same
question, when only one was independently conducted and peer-reviewed. Because the benchmark
was public, there is also no way to know whether the second result reflects a better underlying
system or targeted adjustment to those specific cases, for example through additional training data
or a revised system prompt layered on top of the underlying model. In our ongoing study, of the
40 discrete pieces of evidence we identified for the 23 tools, 39 (97.5%) were authored, funded, or
self-published by the developer whose product was being evaluated, and only one was both
independently conducted and peer-reviewed. Selective publication and outcome reporting may
compound the problem: evaluations with unfavorable results may never be made public, and only
favorable endpoints may be reported, a pattern documented across many areas of medical research.
This is why pre-specification and complete reporting of results, discussed below, matter.19

Moreover, this evidence is frequently reported as a single aggregate accuracy or “passing” figure
against a named benchmark, without specifying which competency the benchmark measures and
without disclosing whether the evaluation was conducted or funded by the developer itself. We
would encourage CDRH to require that any benchmark submitted in support of a marketing
authorization identify, explicitly and in advance (pre-specification), which of the competency
elements described in Section V of the discussion paper it is intended to measure; disclose the
independence (or lack thereof) of the evaluating party, with at least one evaluation of any safetycritical function conducted by an independent evaluator; and justify why performance on that
element is expected to predict safe and effective real-world behavior for the device’s specific
intended use. A benchmarking structure that does not preserve these distinctions risks allowing
sponsors, reviewers, and the public to conflate “passed a test” with “safe to deploy,” which are not
the same claim. To make these distinctions visible, CDRH should expect sponsors to disclose, for
each benchmark: (1) the number of cases and distinct clinical presentations, how cases were
sourced, and their mix by acuity, clinical domain, and population; (2) what counts as a correct
response and who adjudicated it; (3) the scoring rubric and acceptance standards; and (4) the exact
configuration tested, including the foundation model and its version, the system prompt version,
and the retrieval sources. Structured templates already exist for pre-specifying and reporting realworld evidence studies, such as STaRT-RWE,20 and a comparable template for GenAI benchmark
evaluations would serve the same purpose. Benchmark methods and results, including
performance on every prespecified endpoint and not only those that were met, should also be
summarized publicly in the device’s decision summary, so that patients, clinicians, and deployers
can see what was and was not demonstrated.
Benchmark results should also meet the statistical conventions FDA already applies to diagnostic
tests. Endpoints should be prespecified and reported with 95% confidence intervals, consistent
with FDA’s statistical guidance for diagnostic test studies,22 and acceptance criteria for safetycritical behaviors should be set on the confidence bound rather than on the point estimate. Sample
size requirements follow directly. To show with 95% confidence that a critical-error rate is below
1%, a sponsor needs roughly 300 consecutive error-free cases; a perfect score on 64 cases, by
contrast, rules out only a critical-error rate above roughly 4.6%. Where human raters adjudicate
outputs, inter-rater agreement should be reported. Where an LLM is used as a judge, it should itself
be validated against expert human ratings for the specific task before its scores are accepted.
Reporting should follow emerging standards such as the Chatbot Assessment Reporting Tool
(CHART).22 We also encourage FDA to set objective performance criteria for safety-critical
functions such as emergency escalation, with separate criteria for each deployment setting, so that
sponsors’ acceptance thresholds are anchored to a common, publicly known minimum rather than
chosen product by product.
To address the contamination and saturation the paper rightly identifies, we recommend that at
least part of the evidence for any safety-critical competency come from sequestered test sets held
by independent parties and never released to developers. Publishing competency expectations is
valuable because it gives developers a clear floor. But if the test items themselves are public, a
developer can optimize for those items—through targeted training data or system-level
prompting—without improving performance on clinically similar cases that are not on the list.
Sequestration allows FDA to publish what will be tested without publishing the test. Because
evaluating a hosted model necessarily passes test items through the sponsor’s systems, sponsors
should also be prohibited from retaining those items or using evaluation interactions to train or
tune their models. Finally, for devices that use retrieval-augmented generation (RAG), end-to-end
benchmark scores should be supplemented with component testing: whether retrieval returns
relevant and current sources; whether generated content is faithful to what was retrieved; and
whether citations actually support the statements to which they are attached (citation precision). A
correct final answer built on an unfaithful or mis-cited basis is a latent failure that end-to-end
scoring will not detect. Patient-facing devices should also show users the sources behind each
response, so that patients and clinicians can check whether a cited source supports what is said.

IV. Clinical Confirmation: What Medical Education’s Competency Model Gets Right, and
Where the Analogy Breaks (Questions 7, 11, 13, and 15)
We support the CDRH discussion paper’s proposal to draw on the evaluation and credentialing of
human clinicians as an organizing analogy for the competency-based approach described in
Section V, and we agree that device benchmarking followed by clinical confirmation is a
reasonable high-level structure. We would encourage CDRH to draw the analogy more fully,
however, particularly with respect to standardized patient simulation. Clinical trainees are not
evaluated only on written knowledge examinations; they are also evaluated through standardized
patient encounters, such as the Objective Structured Clinical Examination (OSCE), that assess
history-taking, communication, and clinical reasoning under realistic, interactive, and sometimes
ambiguous conditions. We believe the “standardized patient interactions” approach that CDRH
lists among possible clinical confirmation methods in Section V.C deserves to be elevated rather
than treated as one option among several, because it is the closest existing analogue to the
interaction-centric validation we describe above and because it can probe safety-critical
conversational behaviors, such as escalation under user minimization or resistance, that single-turn
benchmarks cannot.

More generally, we would encourage CDRH to organize the clinical confirmation methods in
Section V.C by what each can establish, rather than along a single scale of rigor and exposure.
Retrospective review and simulated cases can speak to analytical and clinical validity;
standardized patient interactions can probe interactive, safety-critical behavior; but only evaluation
with real users can establish clinical utility. For action-directing functions, the relevant question is
what users actually do after an interaction, which requires prospective evidence of user behavior
(for example, whether patients advised to seek emergency care did so), gathered through staged,
early clinical evaluation of the kind described in the DECIDE-AI reporting guideline23 and
informed by FDA’s human factors guidance.14 A recent prospective study of Google’s AMIE
system at Beth Israel Deaconess Medical Center offers a proof of concept: the pre-registered study
deployed a conversational diagnostic AI for pre-visit history taking with real patients, with a
physician supervisor monitoring every conversation in real time against prespecified safety-stop
criteria, and compared its diagnoses and management plans with those of the treating clinicians.24
FDA could expect, or require, staged evaluation of this kind before action-directing functions are
offered to patients without supervision. In response to Question 13, synthetic data may usefully
expand scenario coverage, but it should not substitute for evidence in the real subgroups for whom
performance is in question. Synthetic patients encode assumptions about how a population
presents, which is precisely what subgroup evidence is meant to test. Question 13 also asks about
underrepresented subgroups. Study populations should be generalizable to the device’s intended
users, including people with limited mobility or limited health literacy, who may act differently on
the same advice, and sponsors should report performance separately for such subgroups.25
In response to Question 15, we suggest separating two judgments. For safety-critical failures such
as missed emergencies, acceptance thresholds should be absolute and should not be relaxed in
exchange for benefit elsewhere. For benefit, by contrast, the appropriate comparator is the realistic
alternative available to the user, which is often an unassisted web search, a delayed appointment,
or no advice at all, rather than an idealized clinician. Relatedly, the discussion paper notes that
performance may depend on the type of patient data a device receives. We would encourage CDRH
to expect devices to enforce their validated input scope. A device validated only on text
descriptions of symptoms should be designed to recognize and decline, or appropriately redirect,
inputs outside that scope (such as uploaded images or laboratory reports), and the guardrails that
do so should themselves be validated rather than assumed to work.
At the same time, the medical-training analogy illustrates precisely why competency and validity
are not the same as utility. A second-year medical student may pass the United States Medical
Licensing Examination (USMLE) Step 1, and a graduating student may pass Step 2, yet neither is
thereby granted the judgment or independent authority of a senior resident or attending physician.
The difference is supplied by years of supervised practice with progressively decreasing
oversight—precisely the structure CDRH describes in Section V.A as its inspiration. This
distinction is not merely theoretical. Developer marketing materials in our sample already invoke
the licensing-examination analogy directly, with at least one developer publicly claiming its
system scored 100% on the USMLE, unaccompanied by a published methodology, an independent
audit, or any indication of how that score bears on the system’s actual patient-facing function. The
analogy also breaks in a second way: students sit examinations composed of items they have never
seen, whereas a GenAI system evaluated on publicly released examination questions may have
encountered those questions, or close variants, during training. Any examination-based claim
should rely on unreleased items, ideally administered in partnership with the examining body. We
urge CDRH to make explicit that passing a competency-based benchmark, however wellconstructed, is analogous to passing a licensing examination, and is not a sufficient basis for
autonomous, unsupervised deployment. Consistent with recent proposals for supervised
deployment of clinical AI,26 the scope of a GenAI-enabled device’s authorized function should
expand in graduated steps as evidence of real-world performance accumulates, rather than being
set by benchmark performance alone.

V. Accountability and Lifecycle: Authority Without Accountability (Questions 16, 18, 21,
22, and 24)
A recurring tension in the CDRH discussion paper, and in the broader conversations it cites, is that
GenAI-enabled devices are asked to demonstrate the kind of clinical competency associated with
a licensed, autonomous professional, while remaining regulated as a device rather than as a
practitioner. We believe this tension has an underappreciated consequence. Authority and
accountability have become decoupled in ways that a purely competency-based framework will
not resolve on its own.
A human clinician who diagnoses a patient with, say, a pulmonary embolism and recommends
treatment bears a professional (and in some cases legal) responsibility that extends beyond the
accuracy of the diagnosis itself. If the patient declines the recommended treatment, the clinician is
professionally obligated to counsel the patient on the risks of that decision, to document that
counseling, and, where the patient proceeds against medical advice, to obtain a signed
acknowledgment. This obligation does not depend on whether the diagnosis was correct; it
depends on the clinician’s accountability for the consequences of the interaction as a whole. A
GenAI-enabled device can correctly identify a pulmonary embolism, correctly recommend
treatment, and even correctly communicate urgency, yet it is not held accountable, in any
comparable sense, for a patient’s subsequent decision not to seek care. Freyer et al.,27 whom CDRH
cites in Section V, raise a closely related concern: human clinicians face professional, legal, and
reputational consequences when expected standards are not met, and analogous mechanisms do
not yet exist for GenAI-enabled devices. We would go a step further and suggest that this is not
merely a gap to be filled alongside competency-based evaluation, but a reason for caution in
extending device authority toward action-directing or fully autonomous functions, particularly in
the patient-facing, care-escalation context discussed in Section IV, until accountability
mechanisms of comparable weight are in place. We encourage CDRH to treat the presence or
absence of a credible accountability mechanism, whether contractual, institutional, or regulatory,
as a relevant input to where a given function sits on the activity axis of the two-axis risk framework,
rather than as a postmarket or legal question outside the scope of this paper.
We also encourage CDRH to state explicitly how responsibility is divided between the developer
of a GenAI-enabled device and the organization that deploys it—for example, a health system,
payer, or digital health company that configures the device, supplies its retrieval sources, or
embeds it in a patient portal. Many of the changes most likely to alter device behavior can be made
by either party, and a framework that does not clearly assign responsibility to one of them risks, in
practice, assigning it to neither. Borrowing further from professional licensure, as Bergman et al.
suggest,26 authorization for higher-risk patient-facing functions could incorporate elements with
no analogue in traditional device clearance: time-limited authorization, periodic reassessment
against current benchmarks, and a clear mechanism for suspension when performance falls below
threshold.
In response to Questions 22 and 24, the behavior of a GenAI-enabled device depends not only on
its underlying model but on the layers built around it: the system prompts and instructions added
to every user query, the sources it retrieves from, and the foundation model itself, which may be
updated or deprecated by a third party. A change to any of these layers can alter safety-critical
behavior as much as retraining can. We recommend that such changes trigger re-benchmarking
against the device’s safety-critical acceptance criteria, and that a Predetermined Change Control
Plan (PCCP) specify in advance which changes are anticipated and how each will be verified,
consistent with FDA’s PCCP guidance.28 Material changes should also be visible to users: when a
change alters the underlying model or a safety-critical behavior, the device should notify users and
record the change in a public change log. The public currently has little visibility into these
changes: 3 of 23 tools in our sample published a version history or change log for the product. We
recommend that sponsors of patient-facing GenAI devices maintain a public change log
identifying each material change to the model, system prompts, or retrieval sources, its date, and
whether re-benchmarking was performed and passed.
In response to Question 18, we caution against shifting additional evidentiary weight to postmarket
monitoring for patient-facing triage functions. The characteristic failure of such a function—
falsely reassuring a patient who then does not seek care—is the failure least likely to be observed
after deployment. The patient may never return to the tool, and any resulting harm is recorded, if
at all, in a health system with no link to the interaction. For these functions, postmarket monitoring
is a necessary complement to premarket evidence, not a substitute for it. Where it is used, it should
be ongoing and transparent: sponsors should report safety-critical failures observed after
deployment, including identified under-escalation events, at regular intervals and in a publicly
accessible form. In our sample, no tool received a fully passing score on our outcome/failurereporting transparency criterion, which asks whether the developer discloses an ongoing, postdeployment mechanism for reporting when the tool underperforms, as distinct from a one-time
study snapshot; 15 disclosed partial failure information, and the remaining 8 disclosed none.

VI. Disclosure Infrastructure and Foundation Model MAFs (Question 25)

The CDRH discussion paper seeks feedback on whether voluntary Foundation Model Device
Master Files (MAFs) would be practical and sufficiently useful for premarket review, given that
model developers may have limited incentive to disclose safety-relevant information. We note at
the outset that our data do not measure MAF participation directly. A MAF is a confidential
submission from a foundation model developer to FDA, whereas our landscape review measures
what developers of consumer-facing applications disclose publicly. The findings from our ongoing
work are nonetheless relevant, because they show how developers behave when disclosure is
voluntary, including in circumstances where disclosure would plausibly enhance their reputation.
We scored each of the 23 tools using six FDA-aligned transparency criteria, including data
transparency (whether training or evaluation data composition is disclosed with enough specificity
to assess representativeness). We also recorded whether each tool disclosed basic information
about the model underlying it, corresponding to model-identification items in the CHART22 (items
3–6) and TRIPOD-LLM29 reporting guidelines. Disclosure of the kind of information a Foundation
Model MAF might contain (e.g., model cards, system cards, characterized limitations and failure
modes, or evaluation results on healthcare-relevant benchmarks) does not currently appear to be a
consistent or standard practice among the developers we reviewed, including well-capitalized
developers with otherwise substantial published evidence.
If application developers, who stand to gain reputationally from demonstrating safety, rarely
disclose even which model their product uses, there is little reason to expect foundation model
developers, who face greater commercial sensitivity and less direct clinical accountability, to
participate robustly in a voluntary program. We believe this suggests that a purely voluntary
program may be insufficiently useful without a stronger incentive structure, and we encourage
CDRH to consider whether reference to a Foundation Model MAF, or an equivalent disclosure,
could become an expected component of a complete premarket submission for any device built on
a third-party foundation model, rather than a voluntary supplement to it. At a minimum, sponsors
should identify to FDA the foundation model developer, the specific model and version, how it is
accessed, and the change-notification terms in their agreement with the foundation model
developer. A sponsor needs this information to answer Question 24, and FDA needs it to act on
safety signals: devices that share a foundation model may share its failure modes, which is where
correlated error at scale is most likely to arise, so when a failure is found in one model FDA should
be able to identify every authorized device built on it. In public labeling, sponsors should identify
the foundation model developer and the model version evaluated for authorization, alongside the
public change log recommended above. FDA’s January 2025 draft guidance on AI-enabled device
software functions already encourages model cards in device labeling and public submission
summaries,30 and FDA already requires a software bill of materials for cyber devices;31 disclosure
of a device’s foundation model is a comparable expectation. If FDA concludes that trade-secret
protections limit what it can require, it should say so publicly, so that Congress can consider
granting explicit authority for this transparency.

Conclusion
We appreciate CDRH’s willingness to engage stakeholders on these questions before regulatory
expectations are finalized. Above all, we urge CDRH to state explicitly that GenAI-enabled
products used directly by patients, with or without healthcare professional engagement, require
systematic evaluation in which all three types of evidence (analytical validity, clinical validity, and
clinical utility) are generated and reported. Toward that end, we respectfully urge CDRH to: (1)
clarify that health-specific design features constitute evidence of intended use regardless of
disclaimers, and classify GenAI-enabled functions by the highest-risk function they can reach; (2)
define the clinical interaction, rather than the isolated output, as the relevant unit of validation for
patient-facing conversational devices, and preserve the distinction between analytical validity,
clinical validity, and clinical utility throughout any competency-based framework; (3) add error
detectability, input control, correlated error at scale, and deployment environment as risk
modifiers, decline to treat disclaimers as sufficient mitigation, expect responses to communicate
uncertainty in calibrated language, and require separate, prespecified thresholds for under- and
over-escalation, controlling under-escalation primarily through testing at authorization and at
every model change or recertification, with clinician backup and a prespecified suspension
mechanism for high-risk conversations; (4) require that submitted benchmarks declare the
competency element they measure and the independence of the evaluating party, report
prespecified endpoints with confidence intervals and acceptance criteria set on the confidence
bound, include sequestered, independently held test sets that sponsors may not use for training,
include an independent evaluation of any safety-critical function against objective performance
criteria set for each deployment setting, and disclose how benchmark cases were sourced and
adjudicated, the scoring rubric, and the exact configuration tested; (5) organize clinical
confirmation methods by what they can establish, elevate standardized patient interactions, and
require prospective evidence of user behavior for action-directing functions, from study
populations that include underrepresented subgroups; (6) treat credible accountability
mechanisms, explicit developer and deployer responsibilities, and licensure-like features such as
time-limited authorization as relevant inputs to risk assessment and authorization; (7) require rebenchmarking, user notification, and a public change log when system prompts, retrieval sources,
or the foundation model change, require regular public reporting of postmarket safety-critical
failures, and avoid shifting evidentiary weight to postmarket monitoring for patient-facing triage;
and (8) move toward expected, rather than voluntary, foundation model disclosure, including
identification of the foundation model, its version, and its change-notification terms to FDA, and
of the model developer and evaluated version in public labeling. We would welcome the
opportunity to share the full results of our landscape analysis with CDRH as our manuscript moves
toward publication, and we thank the agency again for the opportunity to comment.

Respectfully submitted,

Angela Casarez, BA (Yale School of Medicine, MD Student)

Rohini Ghosh, MD, MPH (Yale School of Medicine, Postdoctoral Associate)

Therese J. Ziaks, MS, MPH (Yale School of Medicine, Postgraduate Associate)

Kanhai Amin, BS (Yale School of Medicine, MD Student)

Reshma Ramachandran, MD, MPP, MHS (Yale School of Medicine, Assistant Professor)

Sanket Dhruva, MD, MHS (University of California San Francisco, Associate Professor)

Joseph S. Ross, MD, MHS (Yale School of Medicine, Professor)

Use of Artificial Intelligence: In preparing this comment, the authors used Claude Opus 5.5
(Anthropic) in October 2026 to synthesize author notes, revise wording, merge co-author edits,
check sources, and format references. The authors reviewed, edited, and verified all AI-assisted
content, including each reference and data point, and take responsibility for the content of this
comment. This statement follows JAMA Network guidance on disclosing author use of AI.32

References

1. FDA-NIH Biomarker Working Group. BEST (Biomarkers, Endpoints, and other Tools)
Resource. Food and Drug Administration (US); National Institutes of Health (US); 2016.
Updated January 25, 2021. Accessed October 15,
2026. https://www.ncbi.nlm.nih.gov/books/NBK326791/
2. U.S. Food and Drug Administration. Software as a Medical Device (SAMD): Clinical
Evaluation. Guidance for Industry and Food and Drug Administration Staff. December 8,
2017. Adopting IMDRF/SaMD WG/N41FINAL:2017.
https://www.fda.gov/media/100714/download
3. U.S. Food and Drug Administration. Considerations for the Regulation of Generative AIEnabled Medical Devices: Discussion Paper and Request for Feedback. Docket No. FDA2026-N-7874. 2026. Published August 2026. Access October 6, 2026.
 https://www.fda.gov/medical-devices/digital-health-center-excellence/considerationsregulation-generative-ai-enabled-medical-devices-discussion-paper-and-request
4. Wong EYT, Verlingue L, Aldea M, et al. ESMO guidance on the use of Large Language
Models in Clinical Practice (ELCAP). Ann Oncol. Dec 2025;36(12):1447–1457.
doi:10.1016/j.annonc.2025.09.001
5. Kerman H, Siden R, Cool JA, et al. “I double checked it with my own knowledge”:
physician perspectives on the use of AI chatbots for clinical decision-making. J Gen Intern
Med. 2026;41(6):1489-1497. doi:10.1007/s11606-025-10145-0
6. Gottlieb S. A precedent for regulating clinical AI models. JAMA Health Forum.
2026;7(10):e264704. doi:10.1001/jamahealthforum.2026.4704
7. McBain RK, Cantor JH, Breslau J, et al. AI chatbot use and disclosure for mental health
among US adolescents and young adults. JAMA Pediatr. Published online June 3, 2026.
doi:10.1001/jamapediatrics.2026.2015
8. Federal Food, Drug, and Cosmetic Act § 201(h), 21 U.S.C. § 321(h). (2017)
9. Code of Federal Regulations. 21 CFR 801.4, Meaning of intended uses. (2021)
10. U.S. Food and Drug Administration. General Wellness: Policy for Low Risk Devices.
Guidance for Industry and Food and Drug Administration Staff. January 2026.
11. Federal Food, Drug, and Cosmetic Act § 520(o)(1)(E), 21 U.S.C. § 360j(o)(1)(E).
12. U.S. Food and Drug Administration. Clinical Decision Support Software. Guidance for
Industry and Food and Drug Administration Staff. January 2026. Accessed October 6,
2026. https://www.fda.gov/regulatory-information/search-fda-guidancedocuments/clinical-decision-support-software
13. International Medical Device Regulators Forum, SaMD Working Group. “Software as a
Medical Device”: Possible Framework for Risk Categorization and Corresponding
Considerations. IMDRF/SaMD WG/N12FINAL:2014. Published September 2014.
Accessed October 6, 2026.
https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-140918samd-framework-risk-categorization-141013.pdf
14. U.S. Food and Drug Administration. Applying Human Factors and Usability Engineering
to Medical Devices. Guidance for Industry and Food and Drug Administration Staff.
August 2026. Accessed October 6, 2026 https://www.fda.gov/media/80481/download
15. Newgard CD, Fischer PE, Gestring M, et al. National guideline for the field triage of
injured patients: Recommendations of the National Expert Panel on Field Triage, 2021. J
Trauma Acute Care. Aug 2022;93(2):E49–E60. doi:10.1097/Ta.0000000000003627
16. Ramaswamy A, Tyagi A, Hugo H, et al. ChatGPT Health performance in a structured test
of triage recommendations. Nat Med. 2026;32(5):1671-1675. doi:10.1038/s41591-02604297-7
17. Bedi S, Cui H, Fuentes M, et al. Holistic evaluation of large language models for medical
tasks with MedHELM. Nat Med. 2026;32(3):943-951. doi:10.1038/s41591-025-04151-2
18. Wang S, Tang Z, Yang H, et al. A novel evaluation benchmark for medical LLMs
illuminating safety and effectiveness in clinical domains. NPJ Digit Med. 2025;9(1):91.
Published 2025 Dec 26. doi:10.1038/s41746-025-02277-8
19. Dhruva SS, Shah ND, Ross JS. Mandatory Registration and Results Reporting of RealWorld Evidence Studies of FDA-Regulated Medical Products. Mayo Clin Proc.
2020;95(12):2609-2611. doi:10.1016/j.mayocp.2020.04.013
20. Wang SV, Pinheiro S, Hua W, et al. STaRT-RWE: structured template for planning and
reporting on the implementation of real world evidence studies. BMJ. 2021;372:m4856.
Published 2021 Jan 12. doi:10.1136/bmj.m4856
21. U.S. Food and Drug Administration. Statistical Guidance on Reporting Results from
Studies Evaluating Diagnostic Tests. Guidance for Industry and FDA Staff. March 2007.
22. Loder E, Chen X, Guyatt G, et al. CHART Collaborative. Reporting guideline for chatbot
health advice studies: the Chatbot Assessment Reporting Tool (CHART) statement. BMJ
Med. 2025;4(1):e001632. Published 2025 Aug 1. doi:10.1136/bmjmed-2025-001632
23. Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early-stage clinical
evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat
Med. 2022;28(5):924–933.
24. Brodeur PG, Koshy JM, Palepu A, et al. Conversational diagnostic artificial intelligence in
ambulatory primary care: a prospective feasibility study. Lancet. Published online October
8, 2026. doi:10.1016/S0140-6736(26)01535-7
25. Kadakia KT, Rathi VK, Ramachandran R, Johnston JL, Ross JS, Dhruva SS. Challenges
and solutions to advancing health equity with medical devices. Nat Biotechnol.
2023;41(5):607-609. doi:10.1038/s41587-023-01746-3
26. Bergman A, Wachter RM, Emanuel EJ. A Licensure Framework for Autonomous Clinical
AI. JAMA. 2026;335(20):1751-1754. doi:10.1001/jama.2026.5483
27. Freyer O, Wiest IC, Kather JN, Gilbert S. A future role for health applications of large
language models depends on regulators enforcing safety standards. Lancet Digit Health.
2024;6(9):e662-e672. doi:10.1016/S2589-7500(24)00124-9
28. U.S. Food and Drug Administration. Marketing Submission Recommendations for a
Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software
 Functions. Guidance for Industry and Food and Drug Administration Staff. December
2024.
29. Gallifant J, Afshar M, Ameen S, et al. The TRIPOD-LLM reporting guideline for studies
using large language models. Nat Med. 2025;31(1):60-69. doi:10.1038/s41591-024-034255
30. U.S. Food and Drug Administration. Artificial Intelligence-Enabled Device Software
Functions: Lifecycle Management and Marketing Submission Recommendations. Draft
Guidance for Industry and Food and Drug Administration Staff. January 2025.
31. Federal Food, Drug, and Cosmetic Act § 524B, 21 U.S.C. § 360n-2.
32. Flanagin A, Perlis RH, Bibbins-Domingo K. Updated Guidance for Author Use of AI in
Medical Publication. JAMA. 2026;336(11):926-928. doi:10.1001/jama.2026.16613