Noheli Carrasco Torrez

Please see the attached public comment submitted in my individual capacity in response to FDA docket FDA-2026-N-7874, Considerations for the Regulation of Generative AI-Enabled Medical Devices.

My comment addresses Questions 3, 9, 19, and 21, with recommendations concerning patient comprehension, competency evaluation, language-stratified postmarket monitoring, and stakeholder accountability.

Thank you for considering my feedback.

Public Comment on FDA-2026-N-7874
Patient Comprehension as Safety Evidence for Generative AI-Enabled Medical Devices
Submitted by: Noheli Carrasco Torrez, B.S., R.T.(T)
Questions addressed: 3, 9, 19, and 21
Submitted in an individual capacity. Views are my own and do not represent my employer.

Summary of Recommendations
For patient-facing generative AI-enabled medical devices, I recommend that CDRH consider
three complementary approaches: (1) patient comprehension demands as a modifier of risk
assessment (Question 3); (2) task-based comprehension testing with representative intended
users, with the rigor of clinical confirmation proportionate to device risk (Question 9); and (3)
language-stratified postmarket monitoring of communication and comprehension-related safety
outcomes, supported by clearly defined stakeholder roles that preserve manufacturer
accountability (Questions 19 and 21).

Together, these approaches would help distinguish a device's ability to deliver accurate clinical
information from a patient's ability to understand and safely use it.

Clinical Perspective
Midway through her radiation course, a Spanish-speaking patient I treated earlier in my career
looked up at me from the treatment table and asked why she was receiving these treatments.

She came in day after day and lay still under the machine. Yet she still did not understand why.

I could not know exactly where communication had broken down. I knew she needed an
explanation and a connection back to her care team. I explained the purpose of her treatments
and contacted her nurse and physician to request follow-up.

I have been a radiation therapist for nine years, and I have had versions of this conversation
more times than I can count. In my experience, an unanswered question does not always
announce itself. Sometimes patients need help recognizing what they need to know.

That experience informs my response to the FDA's discussion paper, particularly its
consideration of patient-facing risks, competency benchmarking, and postmarket evaluation.

Question 3: Risk Assessment for Patient-Facing Functions
Patient-facing GenAI-enabled medical devices offer important opportunities to expand access to
clinical information. However, delivering information directly to patients can introduce risks
that are not adequately characterized by the accuracy of an output alone.

FDA-2026-N-7874 — Carrasco Torrez Page 1
 Patients should not be presumed incapable of evaluating medical information. The concern is
that recognizing an incomplete explanation, identifying a missing question, or deciding when to
seek help may require clinical knowledge that the intended user does not possess.

A February 2026 Nature Medicine study illustrates this distinction. Researchers evaluated 1,298
adults completing hypothetical medical scenarios with language models or their usual resources.
Tested independently, the models identified relevant conditions in 94.9% of cases, compared
with fewer than 34.5% among participants using those models. For appropriate care disposition,
the corresponding figures were 56.3% for models alone and fewer than 44.2% for participants
using them. Participants assisted by the models performed no better than controls [1].

Although these were English-language hypothetical scenarios involving general-purpose models
rather than FDA-authorized devices, the findings illustrate why model performance alone cannot
establish safe use.

I recommend that CDRH consider patient comprehension demands as a risk modifier for
patient-facing device functions, particularly when misunderstanding could affect symptom
recognition, treatment decisions, or timely access to care.

An output can be clinically accurate yet still contribute to unsafe decisions if patients
misunderstand its meaning or cannot recognize when additional information is necessary. Risk
assessment should therefore consider not only the consequences of incorrect outputs, but also
foreseeable consequences of patients misunderstanding or being unable to act on correct outputs.

Safeguards should include understandable explanations of uncertainty, clear instructions for
clinical escalation, and accessible pathways to qualified human assistance. Where appropriate to
the intended use, devices should also be evaluated for whether they communicate essential
safety information without relying entirely on patients to formulate the right question.

The goal should be to support patient autonomy with information that patients can recognize,
understand, and use.

Question 9: Competency Benchmarking
I support CDRH's consideration of a competency-based approach, particularly the
benchmarking elements described as E.4, Communication quality and user comprehension, and
R.2, Subgroup performance.

Appendix A describes E.4 as addressing whether intended users can understand device outputs,
their limitations, and when escalation is necessary across varied health-literacy levels. This is an
important recognition that communication quality involves more than technically correct
language.

The remaining question is how manufacturers should demonstrate these competencies.

FDA-2026-N-7874 — Carrasco Torrez Page 2
 Expert assessment can establish whether generated medical information is accurate and
linguistically appropriate. It cannot, by itself, establish whether patients understand what the
information means or how to act on it.

A 2025 JAMA Pediatrics study found that GPT-4o produced Spanish translations of pediatric
instructions comparable in quality to professional human translations, as assessed by three
independent professional medical translators [2]. A 2024 Pediatrics study found that ChatGPT-4
and Google Translate performed comparably to professional human translators on assessed
measures for Spanish and Brazilian Portuguese, but showed important shortcomings in Haitian
Creole [3].

Performance depends on the tool, the language, and the task. Success in one setting does not
establish safety in another.

“Seek care if your symptoms worsen” can be translated accurately without helping a
patient recognize which changes require an urgent call.

For patient-facing functions that provide clinically consequential information, I recommend that
CDRH consider task-based comprehension testing with representative intended users,
drawing on established human factors and usability engineering methods, including FDA's
August 2026 guidance and IEC 62366-1, as part of clinical confirmation proportionate to device
risk.

Such testing should examine whether patients can explain the recommended next step, recognize
relevant warning signs, understand the device's limitations, and identify when human assistance
is necessary. Testing should also include realistic interactions in which users provide incomplete
information or do not recognize that additional questions are needed.

Evidence should reflect the languages, health-literacy levels, and clinical tasks within the
device's intended use. Evaluation should involve qualified language professionals and
representative patients where appropriate, rather than relying exclusively on expert review or
automated language metrics.

These approaches need not apply uniformly to all devices. For higher-consequence
patient-facing functions, direct comprehension testing with representative intended users should
be considered before deployment, while lower-risk functions may warrant less intensive
confirmation.

CDRH's consideration of E.4 is an important starting point. Its practical value will depend on
whether the supporting evidence demonstrates understanding by the people expected to use the
device.

FDA-2026-N-7874 — Carrasco Torrez Page 3
 Questions 19 and 21: Postmarket Performance Evaluation and Stakeholder
Responsibilities
I support CDRH's consideration of periodic re-benchmarking, sample-based clinician review,
and performance-degradation monitoring as complementary approaches to evaluating
GenAI-enabled medical devices after deployment.

For patient-facing functions, however, postmarket evaluation should also consider whether
patients continue to understand and appropriately act on the information delivered.

Peter Embi's concept of algorithmovigilance provides a useful foundation for ongoing
evaluation of clinical AI, including attention to real-world effectiveness, equity, and unintended
consequences [4]. Applied to patient-facing GenAI, this approach should extend beyond model
performance to communication and comprehension-related safety outcomes.

I recommend that CDRH consider language-stratified monitoring for devices intended to
communicate clinical information across multiple languages. Appendix A's R.2 element already
recognizes the importance of performance across non-standard dialects, accents, colloquialisms,
and differing literacy levels. These considerations should extend beyond initial benchmarking to
postmarket evaluation of how intended users understand and act on device outputs.

Depending on the device's risk and intended use, relevant measures could include suspected or
documented misunderstandings of device outputs that cause or could contribute to harm,
clinically significant communication errors, failures to recognize escalation needs, and sampled
assessments of patient understanding.

Requests for clarification may provide additional information but should not automatically be
classified as failures. Similarly, referral completion and time to follow-up can be informative,
but transportation, insurance, scheduling, and other barriers must be considered when
interpreting these outcomes.

Overall performance can hide problems affecting smaller groups. Monitoring plans should
therefore identify relevant language subgroups while acknowledging uncertainty when sample
sizes are insufficient for reliable comparison.

Reassessment should follow a risk-based schedule, with additional review triggered by
substantial model or interface changes, deployment in additional languages, modifications to
escalation instructions, emerging communication-related safety incidents, or deterioration in
subgroup performance.

FDA-2026-N-7874 — Carrasco Torrez Page 4
 In response to Question 21, clinicians and healthcare institutions could contribute to postmarket
monitoring by identifying communication-related safety concerns, documenting relevant patient
interactions, and sharing findings with manufacturers through established safety and
complaint-reporting processes. Mandatory Medical Device Reporting requirements would
continue to apply to manufacturers, importers, and device user facilities as appropriate, while
individual clinicians and patients could submit voluntary reports through FDA's MedWatch
program.

These contributions should complement, not replace, manufacturer accountability for evaluating
complaints and implementing device-related corrective actions.

Healthcare institutions may hold information about downstream patient interactions and
outcomes that manufacturers cannot independently observe. Privacy-protective mechanisms for
sharing relevant findings could strengthen postmarket evaluation, provided responsibility for
investigating problems and taking corrective action remains clear.

Manufacturers' postmarket monitoring plans should specify triggers for investigation and
device-related corrective action, while deploying healthcare institutions should retain
responsibility for decisions about continued local use, including pausing affected functionality
when appropriate.

Closing
A system's ability to produce Spanish text is a capability. Whether patients can use that text
safely requires evidence from the people expected to use it.

I can picture my patient with a chatbot that answered every question she typed accurately, in
fluent Spanish.

She still might never have typed the question.

References
[1] Bean AM, Payne RE, Parsons G, et al. Reliability of LLMs as medical assistants for the general
public: a randomized preregistered study. Nature Medicine. 2026;32(2):609–615.
doi:10.1038/s41591-025-04074-y. Publisher correction published April 17, 2026.
doi:10.1038/s41591-026-04404-8.
[2] Ray M, Kats DJ, Moorkens J, et al. Evaluating a Large Language Model in Translating Patient
Instructions to Spanish Using a Standardized Framework. JAMA Pediatrics. 2025;179(9):1026–1033.
doi:10.1001/jamapediatrics.2025.1729.
[3] Brewster RCL, Gonzalez P, Khazanchi R, et al. Performance of ChatGPT and Google Translate for
Pediatric Discharge Instruction Translation. Pediatrics. 2024;154(1):e2023065573.
doi:10.1542/peds.2023-065573.
[4] Embi PJ. Algorithmovigilance—Advancing Methods to Analyze and Monitor Artificial
Intelligence–Driven Health Care for Effectiveness and Equity. JAMA Network Open. 2021;4(4):e214622.
doi:10.1001/jamanetworkopen.2021.4622.

FDA-2026-N-7874 — Carrasco Torrez Page 5