Coalition for Health AI (Lucy Orr-Ewing)

Coalition for Health AI (CHAI)’s Comment (attached)

Dockets Management Staff (HFA-305)​
Food and Drug Administration​
5630 Fishers Lane, Rm. 1061​
Rockville, MD 20852

Re: Docket No. FDA-2026-N-7874, Considerations for the Regulation of Generative
AI-Enabled Medical Devices: Discussion Paper and Request for Feedback

19 October 2026

To the Digital Health Center of Excellence:

The Coalition for Health AI (CHAI) commends FDA for this discussion paper. This is a complex
topic with implications for health care access, quality, cost, and innovation. The invitation for
public commentary facilitates thoughtful, durable, and nuanced regulation that will support
American leadership.

Our process. These comments reflect a structured consultation with our membership. On
October 5, in collaboration with Harvard Law School, we held a briefing and workshop with more
than 40 organizations and individuals from CHAI’s Policy Workgroup. The discussion informed a
member survey issued on October 9, which received 30 responses. Respondents included AI
developers (23%), health systems and providers (23%), academic researchers (18%) and
individual clinicians, including physicians and nurses (16%). The comments below reflect the
input received. Some of the respondents include: Mayo Clinic, New Mexico Medical Board,
American Academy of Family Physicians, Stanford University, Johns Hopkins, University of
Alabama, Ochsner, Penn Medicine, Microsoft, and Infinitus.

1. We support evaluating the deployed system, but the premarket gate remains essential

Members support FDA’s shift to evaluating deployed systems: 76% were supportive (38%
strongly supportive, 38% supportive with reservations), 10% were neutral, 7% concerned and
7% strongly opposed. Members agreed that safety depends as much on prompts, retrieval,
guardrails, configuration and human oversight as on the underlying model.

Most members, though supportive of the overall approach, urged that postmarket monitoring
complement adequate premarket evidence rather than routinely replace it. This concern applies
particularly to tools that materially influence diagnosis, treatment, prescribing, triage or
emergency care. Members cautioned that harm may occur while a device is still being
evaluated, even after launch. Until postmarket networks exist and can demonstrably detect
problems and trigger action, the premarket gate should remain rigorous.

Premarket evaluation should include tests specific to generative AI:
 ●​ hallucination rate and severity;
●​ omission rate and severity, i.e. the failure to include clinically important information such
as a contraindication, red-flag symptom or abnormal result.
●​ consistency across rephrased prompts;
●​ groundedness against source material;
●​ safety behaviour, including recognising safety-critical situations, escalating appropriately
(avoiding both under- and over-escalation), and staying within the device’s intended
scope.

A product that fails risk-appropriate acceptance criteria should not reach the market. Postmarket
monitoring should catch risks that only emerge at scale, not compensate for failures already
known before deployment. Members also asked that reduced premarket evidence requirements
not apply to autonomous or device-controlling functions with irreversible consequences.

Members raised three further points:

●​ Measure outcomes, not just answers. Evaluation should look beyond whether a
device gives acceptable answers to whether sustained use improves outcomes or
causes harm, for patients and for the care teams using it, including effects on
documentation burden, cognitive workload, and time available for direct care.
●​ Validate locally and independently. Health system AI Governance committees typically
see performance data from the manufacturer, measured at other institutions. Members
therefore called for local validation at deploying hospitals - including enough technical
support for deploying organizations to verify transportability and workflow fit - carried out
by qualified, independent third parties as part of postmarket monitoring.
●​ Clarify accountability for patient-facing devices. Members asked who is responsible
when a patient-facing generative AI device causes harm, given that clinicians may have
had little involvement. Authorization should set out three sets of responsibilities:
○​ The manufacturer’s: safe design, validation, change control, monitoring,
complaint handling, escalation logic, corrective action, and notifying deployers of
changes, including those originating in third-party foundation models.
○​ The deploying organization’s: configuration, integration, staff preparation, and
response pathways.
○​ The clinician’s: ordinary professional responsibility where a clinician actually
reviews and acts on an output.
●​ Pre-market evaluation to include economics: The mere presence of a clinician,
nurse, care manager, or on-call service downstream should not transfer responsibility for
defects, inadequate warnings, unsafe escalation design, or foreseeable workload the
device creates. Patient-facing tools can generate uncompensated triage demand and
after-hours work that premarket testing does not reveal. FDA should therefore require
manufacturers to characterize, before authorization, expected escalation volume, acuity
and timing, staffing assumptions, failure modes, and contingency procedures.
2. Postmarket surveillance should be a layered system built on existing infrastructure

Members ranked postmarket surveillance design as CHAI’s top priority (31%) in our comment,
followed by performance across underserved populations (17%), a robust third-party evaluator
ecosystem (14%) and how to measure generative AI performance (14%).

Members agreed that no single mechanism is sufficient. They favored a layered system built on
existing authorities:

●​ Mandate periodic manufacturer reporting. Reporting would be mandatory and scaled
to risk, with scheduled summaries, reassessment after consequential changes, and rapid
escalation of serious signals, akin to PMA post-approval requirements.
●​ Independent third-party evaluators. Evaluators should re-benchmark devices and
review samples at deploying hospitals using a common protocol. Results would go to
both the manufacturer and the hospital, and ideally be shared publicly.
●​ Interoperate with AHRQ’s PSOs as a protected channel for clinician and health system
incident reporting.
●​ Accreditation through CMS deemed-status bodies such as the Joint Commission.
●​ State sandboxes were viewed as valuable for testing methods and supervising early
deployment, but too narrow in population and too fragmented to serve as a national
surveillance model.

Monitoring is only as good as its ability to detect and act on problems. That requires:

●​ a common AI event taxonomy;
●​ version control across models and configurations;
●​ results broken out by setting and subgroup;
●​ reporting routes for patients.

Deployer participation should be protected, funded and voluntary. It should not diffuse
manufacturer accountability or shift uncompensated surveillance work onto hospitals and
frontline clinicians, including physicians, nurses, and other healthcare professionals.

3. Members favor task-specific evaluation over licensure-style credentialing

Most members backed task-specific evaluation over a broad, licensure-style model, which could
imply clinical competence beyond what the evidence shows. Members prefer “authorization” for
specific indications through FDA’s existing intended-use framework, which reaches the same
substance by a different route. Members agreed that competence on one task does not transfer
reliably to another, and that a collection of narrow approvals should not be read as proof of
broad competence.
Any task-specific authorization should:

●​ specify the function, population, setting, degree of autonomy and required clinician
review, attaching to the system as configured and deployed rather than to the foundation
model;
●​ rest on a common safety floor covering escalation, scope adherence, and uncertainty
and deferral, so the device recognizes and declines work outside its scope;
●​ evaluate the full deployed system, including multi-turn behavior, workflow integration and
human-AI team performance, not a static checklist;
●​ be time-limited, with risk-based reassessment after material changes, including
foundation model updates;
●​ remain tied to manufacturer accountability, since devices, unlike licensed clinicians, are
not subject to professional discipline or individual liability.

Members also cautioned against: scope of practice. Device authorizations should not be used
to define or alter the legally established scopes of practice of licensed healthcare professionals,
including physicians, advanced practice nurses, and nurses, or to create artificial distinctions
between generalist and specialist care.

Conclusion

CHAI members support FDA’s direction and see this as a strong start that will need to adapt as
the technology changes. They ask that safeguards be in place before evidence requirements
are relaxed, and that monitoring be able to detect problems and trigger action.

In particular, members ask that:

●​ human oversight is meaningful rather than nominal. Clinician review should count as a
risk control only when the clinician can tell that AI was used, can access the basis for the
output, can evaluate it within the available time and workflow, and can prevent or correct
the action before harm occurs. Where those conditions are not met, “human oversight”
should not be used to justify reduced evidence requirements or to shift manufacturer
responsibilities onto clinicians. Manufacturer responsibilities and clinicians’ professional
responsibilities should remain distinct and clearly defined;
●​ outputs be traceable to their sources. Traceability should let clinicians verify an output
within their workflow before acting on it, not only allow decisions to be audited
afterwards;
●​ performance across populations and care settings - including variance across clinical
roles and workflows - be built into postmarket surveillance from the start, so that gaps in
who benefits do not widen;
●​ FDA recognize that its authorizations may overlap or conflict with state law. A
manufacturer could receive authorization for an autonomous or semi-autonomous
function that some states treat as the unauthorized practice of medicine or another
 profession, or for which they require additional oversight. Members ask FDA to
acknowledge this risk and work with states to reduce it; and
●​ for AI tools outside FDA’s scope, FDA work with ONC, CMS, FTC and state authorities
to clarify who is responsible for what, and confirm that agentic execution does not by
itself bring an otherwise excluded function (e.g., administrative, documentation or claims
work) within device oversight.

CHAI would welcome the opportunity to support FDA as it develops this approach, including by
convening our members and sharing evaluation expertise.

Sincerely,​
Lucy Orr-Ewing, Director of Policy

on behalf of

Coalition for Health AI, Inc.