Loma Linda University School of Medicine (Harine Park)

Commentary from pre-clinical medical students at Loma Linda University School of Medicine

Comments on the FDA Discussion Paper: Considerations for the
Regulation of Generative AI-Enabled Medical Devices

DOCKET NO. FDA-2026-N-7874

Dear Members of the Digital Health Advisory Committee:

As students of Loma Linda University Medical School, we appreciate the opportunity to
comment on the FDA’s Considerations for the Regulation of Generative AI-Enabled Medical
Devices. As a faith-based, non-profit health system with 6 hospitals and more than 16,000
employees, Loma Linda University Health serves more than 1.5 million outpatients every year
and serves as a Level I trauma center for a region covering more than 25% of California. We are
committed to improving patient care through safe, effective, and innovative practices, including
the responsible use of artificial intelligence (AI). The following comments address the proposed
competency-based approach to evaluating GenAI-enabled medical devices, including
risk-based premarket evaluation, standardized benchmarking, clinical confirmation, synthetic
data, independent third-party assessments, multimodal AI capabilities, and postmarket
monitoring to ensure patient safety and device effectiveness.

​
Section V: A Competency-Based Approach for Premarket
Evaluation of GenAI-Enabled Devices

Question 7:
A competency-based approach that combines standardized testing with evaluation in
realistic clinical settings is essential to ensuring the safety and effectiveness of GenAI-enabled
medical devices. Hager et al. suggest that AI models that performed well on medical licensing
exams still struggled with diagnosis, laboratory interpretation, and clinical decision-making when
tested on real patient cases.1 Additionally, Ong et al. highlighted the unique ethical and
regulatory challenges of large language models, emphasizing the importance of comprehensive
evaluation and safeguards before their integration into medical practice.2 These findings
suggest that standardized testing alone may not fully reflect how AI performs in clinical settings.
Clinical confirmation should assess not only an AI device's clinical accuracy and reasoning but
also its ability to recognize uncertainty, communicate effectively with patients and healthcare
professionals, and support safe, patient-centered care.

Question 8:
The two-axis risk framework offers a useful approach to determining the level of
evidence needed before introducing GenAI-enabled medical devices into clinical practice.
Devices that provide general medical information with limited risk of patient harm may require
less extensive testing, such as standardized benchmarking and clinical accuracy assessments.
In contrast, devices that recommend medication adjustments under physician supervision may
require additional testing using real patient cases and evaluations of clinician-AI interactions.
Importantly, physician oversight alone does not eliminate the risks associated with AI-generated
recommendations. Jabbour et al. suggest that systematically biased AI predictions reduced
clinicians' diagnostic accuracy by 11.3%, and providing explanations of the AI's predictions did
not significantly reduce this effect.3 These findings highlight the importance of evaluating how AI
recommendations influence clinical decision-making, even when healthcare professionals
remain involved. For devices that independently initiate high-risk treatments, such as
thrombolytic therapy for acute stroke, more rigorous clinical confirmation, potentially including
prospective clinical studies, would be appropriate. Overall, the two-axis framework could help
ensure that premarket evidence requirements increase with device autonomy and the severity
of potential patient harm while supporting the safe integration of AI into clinical practice.

Question 9:
The proposed benchmarking structure provides a strong foundation for evaluating the
safety and effectiveness of GenAI-enabled medical devices. However, standardized testing
alone may not fully capture how these devices perform in real clinical settings. Accordingly, we
encourage the FDA to strengthen benchmarking expectations by emphasizing:

➢​ Safety (S.1 - S.3): Emphasize testing for hallucinations, recognition of medical
emergencies, appropriate clinical escalation, and communication of uncertainty
throughout multi-turn conversations.
➢​ Clinical Proficiency (E.1 - E.4): Evaluate the accuracy of AI-generated clinical information
against current medical guidelines and assess whether patients and healthcare
professionals can understand and appropriately use its recommendations.
➢​ Generalizability (R.1 - R.2): Require testing across diverse patient populations, language
proficiencies, and clinical settings to identify inconsistent performance and potential
biases.
➢​ Agentic AI Capabilities (A.1): Establish specific safeguards for autonomous clinical
actions, including human approval before high-risk or irreversible decisions.
➢​ Missing Elements: Greater emphasis should be placed on transparency and traceability
of AI-generated information, including verification of clinical sources, cybersecurity risks,
and evaluation of how AI recommendations influence clinician decision-making.

These existing frameworks could reduce unnecessary duplication, promote consistent
evaluation across manufacturers, and strengthen clinician and patient confidence in
GenAI-enabled medical devices.

Question 10:
Sponsors should validate benchmark performance using real-world clinical data and
representative patient scenarios that reflect the device's intended use. Testing should include
diverse patient populations, clinical settings, and complex cases to determine whether
performance remains consistent outside controlled environments.
 Benchmarks should be developed using established clinical guidelines, expert physician
input, and clearly defined performance measures that reflect the clinical competencies being
evaluated. Independent validation should demonstrate that benchmark results meaningfully
reflect clinical reasoning, patient safety, and device performance rather than memorization of
test questions.
Sponsor-developed benchmarks can supplement standardized evaluations by
addressing device-specific functions and clinical risks that existing benchmarks may not
capture. However, these benchmarks should undergo independent review and validation using
sequestered test datasets to minimize bias, data contamination, and overfitting.

Question 11:
Sponsors should select clinical confirmation methods based on the device's degree of
autonomy, intended clinical use, and potential consequences of incorrect outputs. Lower-risk
devices may be evaluated through retrospective patient data or standardized patient
interactions, while higher-risk devices may require prospective clinical studies.
Retrospective evaluations or simulated patient interactions alone may be insufficient for
GenAI-enabled devices that independently diagnose serious conditions, prescribe medications,
or initiate high-risk treatments. These devices may require prospective clinical studies to
evaluate patient safety, clinical outcomes, and performance in real healthcare environments.
Clinical confirmation should include representative patient populations, varying disease
severity, comorbidities, language proficiency, and healthcare settings to reflect the device's
intended use. Testing should also include uncommon but high-risk clinical scenarios to identify
potential failures that may not appear in routine patient encounters.
Human factors and usability studies could help evaluate how healthcare professionals
and patients interpret and respond to GenAI-generated recommendations. Additionally,
supervised clinical pilot programs with predefined safety thresholds could provide valuable
evidence about device performance and clinician-AI interactions before broader implementation.

Question 12:
Sponsors should utilize sufficiently large and representative datasets, with predefined
performance measures and statistical confidence intervals, to evaluate clinical accuracy, safety,
and reliability. Testing should also assess performance across different patient populations and
include uncommon but high-risk clinical scenarios.
Sponsors should compare synthetic datasets with real patient data to identify differences
in demographics, disease prevalence, clinical complexity, and outcomes. Synthetic data should
supplement rather than replace real-world evidence, with appropriate statistical adjustments and
validation to prevent misleading performance estimates.
Benchmarking and clinical confirmation evidence should be combined when they
measure comparable outcomes, use consistent evaluation methods, and demonstrate similar
performance across representative clinical populations. When significant differences exist,
results should be reported separately to maintain transparency and avoid overestimating device
safety and effectiveness.
Question 13:
Synthetic data can be useful when evaluating GenAI-enabled devices in particularly rare
clinical scenarios, medication dosage calculations, and where real-world evidence is limited.
However, synthetic data may provide less reliable significance when evaluating
underrepresented populations, patients with complex comorbidities, and patients from various
cultural and linguistic backgrounds, as generated data may not reflect their real-world clinical
experience.
To mitigate the risk, validated synthetic datasets should be compared against real-world
patient data and include independent clinical expertise to identify potential biases, inaccuracies,
and gaps in representation. Additionally, using independently developed data generators,
conducting subgroup-specific performance evaluations, and reporting synthetic and real-world
results separately can help prevent these reproductions.

Question 14:
Performance comparators and acceptance criteria should be based on clinical guidelines
and clinician consensus. Acceptance criteria should evaluate clinical accuracy, patient safety,
appropriateness of recommendations, and the ability to communicate uncertainty rather than
requiring one exact response.
The standard of care should be established through current clinical guidelines and
consensus among qualified, independent clinicians, while median clinician performance may
serve as a reference for real-world practice. Devices should meet appropriate safety standards
rather than simply matching average clinician performance, particularly when incorrect
recommendations could cause serious patient harm.
The comparator should reflect the device's intended use and the clinical expertise
required to evaluate its outputs. Generalist physicians may be appropriate for routine primary
care functions, while specialist physicians should evaluate devices addressing complex or
high-risk conditions within their specialties.
Human-AI team performance should serve as the primary basis for evaluation when
GenAI-enabled devices are designed to assist healthcare professionals rather than make
independent clinical decisions. Evaluations should compare physician performance with and
without AI assistance to determine whether the device improves clinical accuracy, patient safety,
and decision-making while minimizing the risks of automation bias and overreliance on
AI-generated recommendations.

Question 15:
GenAI-enabled devices should be evaluated against existing clinical practices to
determine whether they provide meaningful improvements in patient care. We recommend
comparing AI-assisted and standard care through:

➢​ Diagnostic accuracy and rates of missed diagnoses.
➢​ Time to diagnosis, treatment, and specialist consultation.
➢​ Frequency of unnecessary testing, referrals, and medical errors.
➢​ Patient outcomes, including adverse events and treatment effectiveness.
➢​ Healthcare accessibility, particularly in rural and underserved communities.
 Comparators can be identified through retrospective chart reviews, clinical workflow
assessments, and real-world healthcare data to reflect the care patients would typically receive
without GenAI assistance. Independent clinician panels and prospective comparative studies
could further justify these comparators by evaluating diagnostic accuracy, specialist referral
delays, and patient outcomes while ensuring that comparisons reflect appropriate clinical
practices.

Question 16:
Yes, independent third parties should be involved in some of the benchmarking and
clinical confirmation, especially in higher-risk GenAI-enabled medical devices. This may improve
objectivity and transparency; however, manufacturers should maintain responsibility for
demonstrating the safety and effectiveness of their devices.
They could provide independent clinical evaluations, standardized benchmarking
datasets, and assess device performance across diverse patient populations. Evaluators should
have relevant clinical and technical expertise, disclose financial conflicts of interest, and remain
independent of device manufacturers.
Third-party evaluations may be challenging when devices require access to sensitive
patient information or proprietary algorithms.
The FDA should encourage transparent evaluation standards, reasonable testing costs,
and multiple qualified third-party organizations to prevent unnecessary barriers for smaller
developers.

Question 17:
Traditional benchmarking methods may not fully capture the performance of multimodal
GenAI-enabled devices because these systems must interpret and integrate multiple sources of
clinical information. For example, a device may accurately analyze medical images and
laboratory results independently but struggle to recognize conflicting findings or appropriately
combine them during clinical decision-making. Therefore, additional testing should evaluate
cross-modal accuracy, clinical reasoning, and performance in realistic patient scenarios. We
recommend additional evaluation criteria for multimodal and predictive AI models, including:

➢​ Multimodal accuracy: Evaluate how accurately AI interprets and combines medical
images, patient histories, laboratory results, and other clinical information.
➢​ Conflicting information: Assess whether AI can recognize and appropriately respond to
conflicting findings from different clinical sources.
➢​ Predictive reliability: Evaluate how accurately AI predicts patient outcomes and
communicates uncertainty in its predictions.
➢​ Patient safety: Assess whether errors in interpreting clinical information could lead to
incorrect diagnoses or treatment recommendations.
➢​ Clinical validation: Test AI devices using realistic patient cases to ensure they perform
safely and effectively in clinical settings.
Section VI: Postmarket Monitoring for GenAI-Enabled Devices

Question 18:
A greater reliance on postmarket monitoring for Gen-AI enabled medical devices may be
appropriate if the uncertainty is manageable and can effectively be monitored during real-world
use. There should still be adequate premarket evidence for the benefit of the device, and the
potential outcomes should not be catastrophic. Postmarket monitoring should supplement
premarket evidence for safety, and not serve as a substitute, ensuring that patient outcomes for
AI-enabled medical device use is measurable and at the very least comparable to those of
non-AI-enabled devices. Protocols should be in place to detect adverse events and address
them in time.
On the two-axis framework of consequences and activity on risk, limited and
non-directive devices might be considered for acceptance. Higher risk devices must be
considered with greater scrutiny, although they may be an appropriate solution for experimental
treatments in which patients have exhausted all other avenues, and are extensively informed
about the risks.
Babic et al. suggest that the vast majority of adverse effects from current AI-enabled
devices arise from a small number of devices, of which most were malfunctions.4 When
compared with non-AI-enabled devices, there were less adverse events for most devices that
were approved. This may be a testament to both the utility of generative AI and current
regulatory practices in place – as such, it may be appropriate to be more stringent with
regulations, not less. For devices with greater premarket uncertainty, it may be appropriate to
impinge on intellectual property during the premarket review process for the sake of greater
transparency.

Question 19:
Re-benchmarking should be encouraged to determine if capabilities have changed after
the premarket evaluation. Clinically representative cases and rare but consequential cases
should serve as benchmarks, along with the devices’ performance across relevant patient
subgroups and clinical settings. A static benchmark cannot anticipate all real-world inputs or
detect unforeseeable failures that emerge after prolonged use. Re-benchmarking would
supplement ongoing monitoring.
Sample-based clinician review would be essential to assess clinical outcomes and
potential harm that may be missed. Statistically defensible sampling should be encouraged, with
deliberate oversampling of high-risk interactions, reported incidents, near misses, & unusual
outcomes.
Concept drift, covariate shift, and algorithmic stability are some examples of what might
lie at the root of adverse events from AI-enabled medical devices. Performance degradation
monitoring should be necessary to detect changes in inputs, patient populations, workflows,and
model behavior. Changes in accuracy, harmful error rates, calibration, clinician corrections,
complaints, and other unexpected downstream actions should be noted. While drift may not
necessitate performance degradation, stable metrics likewise do not prove stable clinical
performance, so clinician oversight along with machine-based supervision may be necessary.
Question 20:
Machine-based supervisory agents that are properly vetted may help scale periodic
re-benchmarking & detecting performance degradation; however, it should not be considered a
guarantee of safety or a substitute for clinical oversight. Its role can be in improving efficiency
and timeliness, although independent verification should be preserved.

Question 21:
All roles including clinicians, healthcare institutions, professional societies,
standards-setting bodies, and manufacturers should be involved in postmarket monitoring.
Clinicians can play an important role when it comes to identifying and reporting incorrect
AI-generated recommendations and potential threat to patient safety in clinical practice.
Healthcare institutions can help establish proper monitoring systems to track real-world data,
identify certain patterns of error, and overall track AI performance. Moreover, professional
societies can provide expertise on specific specialties, establish performance benchmarks, and
more. Standards-setting bodies can focus on AI accuracy and reliability across the healthcare
setting. These roles share many responsibilities, but the role of the manufacturer should remain
central to ensuring continued safety and effectiveness of AI-enabled medical devices.
Manufacturers should be responsible for investigating any reported errors and implement the
necessary corrections or improvements promptly. It is paramount that clear responsibilities and
communication channels be established between stakeholders to allow for collaborative
monitoring without diminishing manufacturer accountability.

Question 22:
​ Post-deployment re-evaluation should reflect the impact on safety and efficacy of the
modifications. The premarket competency-based assessment should serve as a baseline for
performance, limitations, and safety. Re-benchmarking might be scaled depending on changes
in the scope of clinical consequences, intended use, detectability, uncertainty, and reversability.
Factors such as whether the change alters diagnosis, triage, or treatment recommendations, or
whether the changes affect a broad range of the device’s output, actions, or recommendations
should be considered.
Some categories of change that may not significantly affect safety or effectiveness may
include internal analytics that do not alter clinical meaning, interpretation, or output, or
user-interface refinements. Some that may require review may include those that adjust
intended use, clinical implications, or patient populations. Changes that may transition from
clinician-assisted outputs to fully autonomous outputs would need extensive review to minimize
potential risks. PCCPs should consider what evidence is required for continued, equivalent
performance and establish stop criteria for when performance falls below an acceptable
threshold.

Question 23:
Predetermined Change Control Plans (PCCPs) can help for GenAI-enabled medical
devices through establishing a flexible framework, allowing for future modifications while
maintaining patient safety and device effectiveness. PCCP concepts can be adapted by
specifying how the changes will be assessed, even when the exact changes cannot be
predicted. This could mean establishing a review process for replacing the model, changing
system prompts, or updating any retrieved medical information. This would also mean that
testing would need to assess the complete device — testing the final user-facing system —
rather than simply the component being changed since all these components work together.
More specifically, a proposed plan could require each new update to demonstrate that the
device maintains its intended clinical tasks, communicates uncertainty appropriately, and
recognizes any unsafe requests. However, it should also mark changes that fall outside the
plan, such as when adding a new clinical use, which would require a separate assessment of
whether further FDA review is needed. This would help prevent companies from modifying their
devices without unrestricted permission. Finally, through timely progression and the ability to
restore an earlier version, it would help provide additional safeguards.

Question 24:
Manufacturers should be in control when model updates reach patients, instead of
accepting changes automatically from any third-party model developer. However, if changes
were to be initiated, a contract would help by making it so that advanced notices for the changes
be noted and that access to the updated model for testing purposes be allowed. Before
accepting an update, the manufacturer should test the device on its clinical tasks, such as if the
new model makes any changes to medical recommendations. A PCCP could describe which
model updates are covered, the tests and acceptance criteria required, and any changes that
might necessitate further FDA review. A PCCP would help form the framework for defined
modifications and impact assessment. The ability to restore the previous version, should there
be an issue, is important to consider if any safety thresholds are crossed.

Section VII: Other Topics

Question 25:
a.​ Voluntary Foundation Model MAFs can be useful for improving transparency regarding
foundation models that are used in medical devices. They could include the model’s
name, version, intended capabilities, etc…
b.​ Developers should include information about post-training, fine-tuning, and human
oversight. If full disclosure of proprietary information is not feasible, a summary of safety
relevant characteristics might be an alternative.

Question 26:
Yes, agentic GenAI-enabled devices would need additional evaluation since they can
create mistakes that affect future steps before a clinician can intervene. Premarket testing
should evaluate the entire sequence of actions, such as if the tools are being correctly used or if
the device recognizes missing or incorrect information. Acceptance criteria in this should be
much more strict given that errors could cause harm. For example, acceptance criteria could
require the verification of patient identity and withhold an order when information is missing.
Elevated risk actions should require clinician approval, and the devices should be monitored to
track what decisions it takes — even to stop the system when necessary.
Citations:

Hager, P., Jungmann, F., Holland, R. et al. Evaluation and mitigation of the limitations of large
language models in clinical decision-making. Nat Med 30, 2613–2622 (2024).
https://doi.org/10.1038/s41591-024-03097-1

Ong J., Chang S., William W et al. Ethical and regulatory challenges of large language models
in medicine. The Lancet Digital Health 6, e428-e432 (2024).
https://doi.org/10.1016/S2589-7500(24)00061-X

Jabbour S., Fouhey D., Shepard S., et al. Measuring the Impact of AI in the Diagnosis of
Hospitalized Patients: A Randomized Clinical Vignette Survey Study. JAMA. 330(23):2275–2284
(2023). https://doi.org/10.1001/jama.2023.22295

Babic, B., Glenn Cohen, I., Stern, A.D. et al. A general framework for governing marketed AI/ML
medical devices. npj Digit. Med. 8, 328 (2025). https://doi.org/10.1038/s41746-025-01717-9