Azra Bihorac

Please see the attached comment on Docket No. FDA-2026-N-7874, “Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and Request for Feedback,” submitted by Azra Bihorac, MD, MS, as a private citizen.

Comment on Docket No. FDA-2026-N-7874
Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and
Request for Feedback

Submitted by
Azra Bihorac, MD, MS, as a private citizen. Affiliation for identification only: Distinguished Professor of
Medicine and Director, AI for Health Institute, University of Florida College of Medicine. The views
expressed are my own and do not represent the University of Florida or any other institution.
I write as a physician-scientist in critical care and nephrology who develops and evaluates clinical AI. I
disclose that my work includes building and testing patient-facing and clinician-facing AI systems.

Summary
I support the discussion paper's core structure: evaluating the final user-facing device, benchmarking
followed by clinical confirmation, and monitoring after deployment. The competency-based approach
borrows from how clinicians are trained and assessed, and that borrowing carries obligations the paper
has begun to name. I offer three recommendations and two shorter observations. First, the comparator
in Question 14 should be specified, justified and measured, because the clinician reference moves,
partly in response to the devices being evaluated. Second, human oversight should be treated as a
claim that requires evidence before it lowers assessed risk. Third, evaluation should test whether a
device elicits and respects the goals of the person receiving care. I also comment briefly on change
control (Questions 22 to 24) and independence (Question 16).

1 The comparator moves (Questions 14, 15 and 19)
Section V.D proposes comparing a device with "a panel of qualified clinicians whose consensus reflects
the applicable standard of care, or ... a median clinician in practice," and asks in Question 14 how the
standard should be defined and justified. I recommend four things.
1. Name the comparator and the reason for it. A sponsor should state whether the reference is the
standard of care or the median of practice, and why that reference fits the intended use. The two
can differ substantially, and a device that matches the median of practice may fall short of the
standard of care.
2. Measure the reference, at baseline and over time. Where the median clinician is the comparator,
the unaided performance of clinicians in the intended setting should be measured at premarket
evaluation and again during re-benchmarking (Question 19). The January 2025 draft guidance on
AI-enabled device software functions already asks whether the user with the device performs "the
same or better than the operator alone." Neither document treats the operator's own performance
as something that can change. If sustained use of a device erodes independent skill, the reference
falls and the device appears to improve without changing.
3. Report the team against the better of its members. A meta-analysis of 106 experiments found
that human-AI combinations, on average, performed worse than the better of the human or the AI
alone (Hedges' g = -0.23), and fell below the machine when the AI was the stronger party (g =
-0.54), even while improving on the human alone (Vaccaro et al., 2024). Reporting only
improvement over the unaided clinician can therefore hide a system that performs below the device
alone. Question 14's human-AI team comparison should report both.

Bihorac, comment on FDA-2026-N-7874 · 1
4. Describe competency as evidence, not qualification. Borrowing the logic of clinical assessment
is useful, but the order matters. A physician reaches the licensing examination after years of
supervised practice, evaluated by many teachers on reasoning, integrity and accountability as well
as knowledge, and the examination is read against that record. For a model, the examination would
come first and stand in for it. A clinician who passes still practices within licensure, supervision, peer
review and the duty owed to each patient; a device that passes a benchmark carries none of these.
I recommend that the final guidance describe benchmark and clinical performance as evidence that
a device performs its intended use, and avoid language suggesting that a device is qualified or
credentialed to practice. That vocabulary will travel beyond the agency.

2 Oversight is a claim that needs evidence (Questions 1, 4, 18 and 20)
The paper relies, appropriately, on clinicians and supervisors to notice errors. Recent evidence shows
how fragile that reliance can be.
In a participant-blinded randomized experiment, 225 physicians from 42 countries made neurocritical
care decisions with a conversational model. When the model was covertly instructed to steer toward
harmful options, harmful final decisions rose from 2.0% to 49.4%; 141 of the 190 physicians who made
a harmful decision reported nothing unusual, and clinical experience conferred no clear protection
(Barrit et al., 2026, preprint). Physicians' trust tracked their agreement with the model rather than its
correctness. In a procurement evaluation of ambient scribes, physicians rated highest the system that
made more errors on audit (Weiss et al., 2026). In the first prospective study of a patient-facing
diagnostic conversational system, every conversation was supervised live, and a harm surfaced only
later, at the clinical visit (Brodeur et al., 2026; Omar and Nadkarni, 2026).
1. Treat supervision as a risk modifier only when its performance is shown. In the risk framework
(Question 1), the presence of a clinician or supervisor should lower assessed risk only to the extent
that the sponsor demonstrates detection of consequential errors under realistic conditions, including
fluent, confident and adversarially steered outputs.
2. Test intended users, not titles. For generalist and specialist users (Question 4), evaluate whether
the intended users recognize consequential failures. Professional title alone does not demonstrate
that ability.
3. Earn reduced premarket evidence with demonstrated response. Greater reliance on postmarket
monitoring (Question 18) should depend on demonstrated capacity to detect, investigate, pause
and correct, with defined response times. Later detection cannot reverse every harm.
4. Test supervisory agents for shared failure. Machine-based supervisors (Question 20) should be
evaluated for correlated errors with the primary system, and for what happens when the two
disagree.

3 Whose goals govern (Questions 3, 9 and 26)
Appendix A treats empathetic communication, avoiding manipulation and user comprehension as
assessable behaviors. These are meaningful protections. They do not establish whose goals govern a
recommendation. Consider a seriously ill person who values remaining lucid and at home over a
treatment that may prolong life. A device can explain the options accurately and warmly and still steer
toward the treatment. The question for evaluation is whether it elicits the person's priorities, preserves
them across a conversation, respects refusal and changed minds, and connects the person with
continuing human care. We cannot mistake superior knowledge for benevolence.

Bihorac, comment on FDA-2026-N-7874 · 2
1. Add goal-concordance to the benchmark elements (Question 9). Include tests of eliciting and
respecting expressed patient goals, honoring refusal, and offering access to human review.
2. For patient-facing functions (Question 3), evaluate comprehension and meaningful refusal
together, and measure whether escalation reaches a person able to respond.
3. For agentic devices (Question 26), require a goal-concordance check at the human-oversight
checkpoints the paper already proposes before irreversible actions, and evaluate safe handoff and
continuity when the system stops.

4 Change control (Questions 22 to 24)
In the prospective study cited above, the system's base model was changed partway through
enrollment, which the investigators reported transparently and which did not count as a protocol
deviation because the protocol did not name a model (Brodeur et al., 2026). I recommend that
evaluation protocols, and any predetermined change control plan, specify model versions, define what
change triggers re-benchmarking against the same comparator, and preserve the ability to pause or
revert, including for changes initiated by third-party model developers.

5 Independence (Question 16)
I support structurally independent adjudication. Independence should extend beyond adjudication to the
design, analysis and reporting of evaluations, and conflicts should be disclosed at each of those stages.

Conclusion
The paper's analogy to clinical training is its strength. Clinicians are assessed through years of
supervised practice, measured against a standard of care, and held accountable to the people they
serve. A framework built on that analogy should measure the reference it borrows, demonstrate the
oversight it relies on, and test whether the device serves the goals of the person in its care. I would
welcome the opportunity to discuss these recommendations.

References
1. Brodeur PG, Koshy JM, Palepu A, et al. Conversational diagnostic artificial intelligence in ambulatory primary care: a
prospective feasibility study. Lancet. Published online October 8, 2026. doi:10.1016/S0140-6736(26)01535-7
2. Omar M, Nadkarni GN. Conversational diagnostic AI in primary care: what happens after it speaks? Lancet. Published online
October 8, 2026. doi:10.1016/S0140-6736(26)01763-0
3. Barrit S, Salvagno M, Corriero A, et al. Subversion of clinical judgment by conversational artificial intelligence. medRxiv.
Posted October 7, 2026. doi:10.64898/2026.10.05.26364541 (preprint, not peer reviewed)
4. Vaccaro M, Almaatouq A, Malone T. When combinations of humans and AI are useful: a systematic review and metaanalysis. Nat Hum Behav. 2024;8:2293-2303. doi:10.1038/s41562-024-02024-1
5. Weiss E, Zsenits B, Nasar F, et al. Comparative evaluation of ambient digital scribe systems in clinical documentation. J Am
Med Inform Assoc. Published online 2026. doi:10.1093/jamia/ocag157
6. US Food and Drug Administration. Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and
Marketing Submission Recommendations. Draft guidance, January 7, 2025. Docket FDA-2024-D-4488.

Bihorac, comment on FDA-2026-N-7874 · 3