Yann Sakref

See attached comment responding to Discussion Questions 16 and 21.

Comment on Docket FDA-2026-N-7874: Independent Third Parties and Local
Validation of GenAI-Enabled Devices
Re: Docket No. FDA-2026-N-7874, Considerations for the Regulation of Generative AI-Enabled Medical Devices:
Discussion Paper and Request for Feedback (August 2026). Response to Discussion Questions 16 and 21.

Oct. 19, 2026

Yann Sakref, PhD, Postdoctoral Scholar, Department of Surgery, Stanford University
Anthony Abou Mrad, MD, Postdoctoral Scholar, Department of Surgery, Johns Hopkins University
Jeffrey K. Jopling, MD, MSHS, Assistant Professor, Department of Surgery, Johns Hopkins University
Lisa Marie Knowlton, MD, MPH, FACS, FRCSC, Associate Professor, Department of Surgery, Stanford University

To the Digital Health Center of Excellence (DHCoE):

Thank you for the opportunity to comment on this discussion paper. We are clinicians and researchers in academic
medical centers, and we respond to Questions 16 and 21 from the point of view of the healthcare institutions where
GenAI-enabled devices will be deployed.

In summary, we support the involvement of qualified, independent third parties in the competency-based approach.
We particularly identify a significant opportunity for their involvement in postmarket monitoring to assess the
performance of devices within the institutions that implement them. Before authorization, third parties can assist in
device benchmarking and clinical confirmation for FDA review. Post-authorization, multiple accredited third parties
could conduct local validation, evaluating devices within their deployment environments. We believe this localized
evidence could expedite AI adoption in hospitals, providing manufacturers with a standardized approach to gather data
relevant to specific settings. Each point is elaborated below.

Premarket evaluation provides a reasonable assurance of safety and effectiveness for the intended use. From a
hospital's perspective, it is crucial to assess the device within its own environment for two key reasons. First, hospitals
need transparent and independent performance data. In our experience, when an AI-enabled device is presented to a
hospital governance committee, the majority of performance data originates from the manufacturer and is frequently
based on assessments conducted at different institutions. This scenario raises the potential for bias in the evaluation
process. We concur with the paper’s emphasis on the importance of independence between development and testing,
believing hospitals would also significantly benefit from this approach. Second, performance does not automatically
transfer from one hospital to another. Patient populations, documentation practices, and clinical workflows differ across
institutions, and the outputs of a GenAI-enabled device depend directly on these local inputs. Its performance should
therefore be measured where it is used.

In practice, a third party could implement the proposed post-market monitoring methods at each hospital.
Independent clinicians would review a sample of past cases to establish a local evaluation set. With each model update,
the device would be re-benchmarked using this set, augmented with recent cases to track performance over time.
Results would be reported to both the hospital and manufacturer, who could outline this process in its monitoring
program. The Center for Devices and Radiological Health (CDRH) could consider this information when deciding upon
acceptable levels of premarket uncertainty. The manufacturer would remain accountable for its device, and the
regulatory decision would remain with FDA.

Many hospitals have yet to conduct this type of evaluation independently, presenting an opportunity for third-party
assistance. A national survey of 2,174 nonfederal US hospitals found that only 46.7% reported conducting all three local
evaluation practices (accuracy evaluation, bias evaluation, and post-deployment monitoring) for most or all of their
predictive AI models [1]. Local evaluation requires expertise and infrastructure that not all hospitals possess. Other
sectors rely on expert independent evaluators adhering to a common standard, such as the independent audit of public
companies [2] and SOC 2 examinations of service organizations [3]. In those sectors, several firms offer the service under
the same standard, which helps it scale. An accreditation scheme similar to ASCA, open to several third parties and with
evaluation methods or datasets qualified as MDDTs, could effectively organize and scale this process.

The third parties must be independent and clinically qualified. They should be structurally independent from the
manufacturer. Adjudicators should be qualified in the clinical domain of the device. Since these third parties may be
compensated by either the manufacturer or the institution, it is essential to disclose the source of payment; their fees
should not be contingent on the findings, and their accreditation must be subject to audit and potential withdrawal. We
would exclude third parties from early development phases and would not anticipate local validation for lower-risk
functions within the two-axis risk framework.

We believe third-party local validation could accelerate clinical AI adoption by standardizing device evaluations across
hospitals and health systems. Without a common approach, each local validation can be a barrier, necessitating
separate pilot studies with distinct protocols and data use agreements. A third party would need to establish a data
agreement upon its first evaluation at a hospital; however, subsequent evaluations could follow the same protocol for
all new devices. As the third party partners with more hospitals and develops the requisite data and legal infrastructure,
local evaluations would become more efficient, particularly benefitting hospitals without their own evaluation
infrastructure.

Thank you for considering these comments. We would be glad to provide further details.

Contacts: ysakref@stanford.edu, drlmk@stanford.edu

References

1. Everson J, Nong P, Richwine C. Uptake of Generative AI Integrated With Electronic Health Records in US
Hospitals. JAMA Network Open. 2025;8(12):e2549463. doi:10.1001/jamanetworkopen.2025.49463
2. Sarbanes-Oxley Act of 2002, Pub. L. 107-204, section 101 (15 U.S.C. 7211), establishing the Public Company
Accounting Oversight Board.
3. AICPA. SOC 2 Reporting on an Examination of Controls at a Service Organization Relevant to Security,
Availability, Processing Integrity, Confidentiality, or Privacy. AICPA Guide, updated October 15, 2022.