Duke Health (Jeffrey M. Ferranti)

See attached comment from Duke Health.

Office of the Executive Vice President
Jeffrey M. Ferranti, MD, MS
Executive Vice President & Chief Digital AI, and
Transformation Officer
Professor of Pediatrics and Informatics
Duke Health

October 14, 2026

Response to the Docket No. FDA-2026-N-7874 for “Considerations for the Regulation of Generative AIEnabled Medical Devices”

Thank you for the opportunity to provide comments and feedback on Request for Information: Considerations
for the Regulation of Generative AI-Enabled Medical Devices. As one of the nation’s leading academic health
systems, we appreciate the opportunity to offer our input as the Agency considers future regulations around
generative artificial intelligence (GenAI)-enabled medical devices.

Duke Health integrates the Duke University Health System (all of our hospitals, clinics, and the Duke Health
Integrated Practice), Duke University School of Medicine, Duke- National University of Singapore Medical
School, and the Duke University School of Nursing. It also incorporates the health and health research
programs within the Duke Global Health Institute as well as those in schools and centers across Duke
University, including the Duke Robert J. Margolis Institute for Health Policy, the Duke Clinical Research
Institute, the Duke Clinical Research Unit, and the Duke Health AI Evaluation and Governance program.

Based on our experience, Duke Health offers the following considerations for GenAI-enabled
medical devices:

Considerations for the Assessment of Risk for GenAI-Enabled Devices
Discussion Question Response:
1. Does the two-axis risk framework, organized around device activity and the consequence of
relying on an incorrect output, appropriately capture the dimensions most relevant to the risk
of a GenAI-enabled software function? If there are additional dimensions—such as the
reversibility of a resulting action, the availability of downstream safeguards, the time
pressure of the deployment setting, or the traceability of the output (i.e., to primary source
materials)—that should be represented in a risk framework, please describe and provide
examples of how they should be represented.

Response:

The two-axis risk framework is appropriate; however, both axes require definition to be
effectively used. A rating of low, moderate, or high risk should be accompanied by a clear
justification. For example, when rating patient safety risk, Duke Health has adopted a
specific set of contributing factors. These include local context, expertise, and infrastructure
in addition to the kinds of technology and workflow considerations highlighted in the
discussion paper. Examples suited to the “consequences” axis could include:
 • Proximity to clinical decision-making and direct patient care
o Direct: AI that influence the diagnosis, treatment, or prevention of a disease or
condition
o One degree separated: AI that influence Insurance pre-approval for treatment; AI
that support patient access to care (e.g. appointment scheduling); AI that support
communication, documentation, or outreach with individual patients
o Two degrees separated: For operational or planning purposes, not with
individual patients
• State of the healthcare situation: critical, serious, non-serious, or multiple/variable
• Detectability of adverse consequences arising from AI use and the reversibility of
incorrect AI-assisted decisions.
• Breadth of the use case and population of patients whose care is affected
• Implementer’s capacity to undertake an evaluation (expertise, infrastructure,
experience)

When characterizing the level of action-directiveness, specific contributing factors could
improve the quality of the assessment and guard against unintended consequences, such as
proliferation of disclaimers and other protective language that reduces liability without
benefit to safety. Possibilities include:
• Experience level of the intended AI user and his/her self-efficacy to override AI’s
recommendations
• AI’s capability to flag recommendations with greater uncertainty or conflicting
supporting evidence
• End users’ ability, inclination, and time to independently review the source materials
and evidence supporting AI recommendations
• Urgency/time-sensitivity of the decision or action taken with AI assistance
• Potential of the AI tool to contribute towards automation bias, degradation of human
vigilance, and clinical deskilling over time.

At Duke Health, defining the specific factors that contribute to a global assessment of risk
provides sufficient context to justify decisions while preserving the flexibility necessary to
respond to a continuously evolving landscape of technology and adoption.

A Competency-Based Approach for Premarket Evaluation of GenAI-Enabled
Devices Discussion Questions Response:
7. Is the competency-based approach described above, i.e., device benchmarking followed by
clinical confirmation, a useful and appropriate framework for evaluating GenAI-enabled
devices?

Response:

Meaningful evaluation begins with defining the use case. Generative and agentic AI systems
are no exception and should not be approved for blanket use. FDA should think about AI
systems as machines that accomplish specific tasks. Testing approaches, metrics, and
approvals and clearances will need to be tailored accordingly.
 For a defined use case, the phased approach of benchmarking followed by clinical
confirmation is reasonable; however, the “competency-based” framing is problematic.
Clinician training relies on social and cognitive processes that computers do not replicate.
Even when machine output resembles or exceeds that of human clinicians, it is derived from a
process that does not resemble human learning and is subject to different failure modes.
Anthropomorphizing AI systems risks creating blind spots for evaluation and, ultimately,
safety. Effective evaluation design reflects an understanding of the mechanisms that produce
the output and the ecosystem in which the output will be used.

8. How might the two-axis risk framework described in Section IV be considered within the
competency-based approach described above to help determine the level of evidence needed
for a premarket submission?

Response:

The type and rigor of evaluation should align directly with the two-axis risk framework. In
general, tools with at least moderate risk should undergo in vivo testing and validation in
real-world clinical settings. Many responsible AI vendors already share testing and validation
results with prospective customers upon request. To reduce friction and strengthen
expectations for transparency, FDA should establish a public evaluation repository or require
vendors to make this information readily available to prospective and current customers.
Disclosures should include high-level descriptions of the setting, outcome measures, and
aggregate results, while allowing vendors and health systems to keep detailed protocols
confidential.

Most responsible AI vendors already conduct early testing and validation through early
customer pilots or early adopter programs. FDA should continue to permit these activities
without imposing undue burdens on vendors and healthcare organizations working together
to ensure that new AI products are safe and add value to health care. FDA could further
support this work by drawing on prior vendor–health system collaborations and establishing
safe harbors or regulatory sandboxes for testing new AI products, in exchange for making
pilot results available to prospective customers.

Postmarket Monitoring for GenAI-Enabled Devices Discussion Questions
Response:
18. CDRH is considering whether it may be appropriate to accept greater premarket uncertainty
regarding a GenAI-enabled device’s benefit-risk profile through greater reliance on postmarket
monitoring. Under what conditions might such an approach be appropriate, and what
characteristics of a monitoring program would need to be in place to justify reduced premarket
evidence? Are there device types or risk profiles for which this approach would not be
appropriate?

Response:

Post-market monitoring of generative and agentic AI systems presents many of the same
challenges as premarket evaluation. We commend CDRH for treating premarket evaluation
and post-market monitoring as a continuum.
 A central evaluation repository or comparable transparency mechanism—described in
response to Question 8 for initial real-world testing—should also support FDA’s post-market
monitoring strategy. At a minimum, vendors should disclose to current customers their highlevel methods and results for regression testing using a reasonably sized library of realistic
test cases. This transparency is especially important after substantial software changes or
replacement of the underlying large language model.

FDA should also encourage vendors to publish evaluations conducted by healthcare
organizations during the post-market phase. FDA is uniquely positioned to assess or audit the
quality of this evidence, giving customers reasonable confidence in vendor-reported results
and reducing the need for healthcare organizations to repeat evaluations already performed by
peers.

Premarket testing cannot guarantee safety because agentic and generative AI systems produce
undesirable, inaccurate, or harmful outputs in non-deterministic ways. Paradoxically, as AI
quality improves, its errors may become harder to detect. We therefore encourage FDA to
establish a voluntary safety and adverse-event reporting program for agentic and generative
AI, with safe-harbor protections that allow healthcare organizations and individual users to
report concerning events despite vendor-contract gag clauses. 1 FDA should also adapt
MedWatch to incorporate elements relevant to the generative and agentic AI ecosystem.

Closing
Thank you for the opportunity to provide these comments. Duke Health’s faculty, researchers, and physician
scientists bring deep expertise across clinical care, health policy, and biomedical innovation, and have
regularly served as trusted advisors to both the Executive Branch and Congress. We welcome continued
engagement and stand ready to support FDA as it considers policies surrounding future regulations around
GenAI-enabled medical devices.

Sincerely,

Jeffrey M. Ferranti, MD, MS
Executive Vice President & Chief Digital, AI, and Transformation Officer
Professor of Pediatrics and Medical Informatics
Duke Health
Ratwani RM, Hodgkins M, Bates DW. Improving Electronic Health Record Usability and Safety
Requires Transparency. JAMA 2018;320(24):2533-2534. DOI: 10.1001/jama.2018.14079.