Xu Shi

Please see attached.

Xu Shi, PhD
Associate Professor of Biostatistics
Department of Biostatistics
University of Michigan
1415 Washington Heights
Ann Arbor, MI 48109
Email: shixu@umich.edu

October 18, 2026

Dockets Management Staff (HFA-305)
Food and Drug Administration
5630 Fishers Lane, Rm. 1061
Rockville, MD 20852

Re: Docket No. FDA-2026-N-7874: “Considerations for the Regulation of Generative
AI-Enabled Medical Devices: Discussion Paper and Request for Feedback”

Dear Dockets Management Staff,
Thank you for the opportunity to provide comments. I submit these comments in my personal academic
capacity and draw on my experience as a biostatistician working in causal inference, negative control
methods, real-world data, and postmarket surveillance. The views expressed are my own and do not
represent any institution or organization.
I focus on Question 19 and have three related considerations. First, CDRH could consider how the established
real-world surveillance infrastructure and experience of Sentinel and CBER’s BEST system might complement
the NEST ecosystem. Second, when the goal is to evaluate clinical benefit or harm, a counterfactual framework
and causal inference methods can help define and interpret the relevant clinical comparison. Third, because
observational postmarket analyses may remain vulnerable to bias and other systematic errors, negative controls
may provide useful tools for validity assessment and bias diagnosis.
1. Build on existing real-world surveillance infrastructure and experience. One opportunity may
be to draw on established real-world safety surveillance capabilities and experience in postmarket evaluation
of GenAI-enabled devices. CDRH already uses Sentinel, where applicable, to complement other sources of
device-safety evidence.[1] Sentinel’s distributed infrastructure, common data model, data-quality processes, and
experience in active surveillance support longitudinal analyses of real-world data, while CBER’s BEST system
provides related experience in biologics and vaccine safety and effectiveness surveillance.[2, 3] These capabilities
could complement NEST’s device-specific expertise and collaborative ecosystem.[4] Their potential value is not
simply access to additional real-world data, but the infrastructure, operating experience, and methodologies
needed for longitudinal, distributed, and active surveillance. For AI-enabled devices, applying these capabilities
may require linkage to device-specific information on deployment, version, and actual use. More broadly, the
key question is whether the resulting data are fit for purpose, including whether they can adequately identify
exposure to the AI-enabled device, the comparator, clinical outcomes, and major sources of confounding and
measurement error.
2. Use causal inference when the monitoring question concerns clinical benefit or harm. FDA’s
2025 request for public comment emphasized real-world evidence for post-deployment AI monitoring and asked
specifically about data sources.[5] A natural next question is how to use these data to evaluate clinical benefit
or harm rather than only detect changes in model performance. Real-world data provide the observations,
but causal inference helps determine what clinical comparison those observations can credibly support. A
counterfactual framework can make explicit the target population, device version and setting, what constitutes
device use, and what care would otherwise have occurred. Whether clinicians view, modify, or act on AI outputs,
and how those behaviors change over time, may be central to interpretation. The same framework also draws
attention to whether the clinical outcome is observed consistently in routine data: outcome definition and
validation, testing and follow-up intensity, informative visiting and observation processes, reporting delays,
missingness and loss to follow-up, and changes in data-source coverage or processing can all affect observed
trends. One practical option may be a prespecified causal monitoring protocol that states the clinical comparison,
exposure ascertainment, outcome definition, major sources of confounding and measurement bias, and planned
sensitivity or bias analyses.
3. Use negative controls for bias diagnosis when appropriate. Even after the clinical comparison is
carefully defined, observational postmarket analyses may remain vulnerable to residual confounding, selection
bias, and other sources of systematic error. Negative controls, when scientifically appropriate, can provide one
useful diagnostic for such biases. They can help assess whether an observed signal reflects a true device-associated
benefit or harm or instead reflects systematic bias. As part of its PDUFA VII commitments, FDA has supported
methods development on negative controls to improve causal inference and assess validity in non-interventional
RWE studies.[6]
I would welcome further discussion with FDA on how experience in causal inference and drug and vaccine
surveillance could contribute to post-deployment monitoring of GenAI-enabled devices.

Respectfully submitted,

Xu Shi, PhD
Associate Professor of Biostatistics
University of Michigan

References
[1] Sentinel Initiative. Devices & Radiological Health Studies. Accessed October 18, 2026.
[2] Sentinel Initiative. How Sentinel Gets Its Data. Accessed October 18, 2026.
[3] U.S. Food and Drug Administration. CBER Biologics Effectiveness and Safety (BEST) System. Accessed October 18, 2026.
[4] U.S. Food and Drug Administration. National Evaluation System for health Technology (NEST). Accessed October 18, 2026.
[5] U.S. Food and Drug Administration. Measuring and Evaluating Artificial Intelligence-enabled Medical Device Performance in
the Real-World: Request for Public Comment. 2025.
[6] Duke-Margolis Center for Health Policy. Understanding the Use of Negative Controls to Assess the Validity of
Non-Interventional Studies of Treatment Using Real-World Evidence. FDA Convening, March 8, 2023.