Adams Healthcare AI Hub (Emmanuel Adams)

See attached :
Emmanuel Adams Response to FDA Discussion Paper on Considerations for the Regulation of Generative AI-Enabled Medical Devices.

Response to FDA Discussion Paper: Considerations for the Regulation of Generative AI–
Enabled Medical Devices

Docket No. FDA-2026-N-7874

Commenter: Emmanuel Adams, Esq., BCC1, & Founder, Adams Healthcare AI Hub - https://
adamshealthcareaihub.com/ 2

Perspective: Attorney, Clinical Chaplain, and Advocate for Responsible Healthcare AI

Section A: Discussion Question 3

CDRH seeks input on whether and when a GenAI-enabled function that results in delivery
of clinical information to patients, as opposed to HCPs, could present different or higher
risks, while also recognizing the potential benefits associated with improved patient
empowerment, engagement, and access to clinical information. What device characteristics,
output features, or safeguards might mitigate risks that could arise when a user lacks the
domain knowledge to independently evaluate an output, without unnecessarily
underestimating patient capability?

At adamshealthcareaihub.com, we focus on and advocate for a Two-track platform for healthcare
AI: the patient track and the healthcare practitioner track. These two tracks capture the two
foundational elements in healthcare: the patient and the care provider.

Whether and when a GenAI-enabled function that results in delivery of clinical
information to patients, as opposed to HCPs, could present different or higher risks?

Yes, a GenAI-enabled function that delivers clinical information to patients, as opposed to HCPs,
could present different or higher risks.

Reasoning: Because of the constitutive differences between patients and HCPs in the context of
healthcare. For example:

1. The patient is more vulnerable because it is the patient who needs the care. The patient
seeks care, and the HCP provides it. So the patient is at the receiving end. If the clinical
information is wrong in any way, the patient is exposed to higher and actual physical risk. The
HCP may be exposed to other forms of risk, such as liability, in the healthcare ecosystem.

1 Board-Certified Clinical Chaplain at the University of California, San Francisco Medical Center (UCSF)

2 An Advocacy platform for responsible healthcare AI policy and dual-track, simple, concise education on what healthcare
diagnostic AI is, how it is used in patient care, and patients’ and practitioners’ rights and liabilities under federal and state law in
America. The platform recommends a governance and policy framework template for responsible pre- and post-deployment
auditing of healthcare AI and offers a free Healthcare AI Governance Policy Guide for Rural Healthcare Systems in America.

Page 1 of 5



 2. Epistemic difference also makes up the patient/HCP constitutive differences. The HCP is
most of the time more medically literate than the patient (Except when the HCP becomes a
patient). This implies that ordinary patients will have limits in their understanding of some
medical terminology or language. Differences in medical literacy can make GenAI-enabled
delivery of clinical information to patients, rather than HCPs, present different or higher risks.
This also implies that even when GenAI-enabled systems deliver inaccurate clinical information,
e.g., hallucinations, the HCP’s medical literacy can serve as a barrier to the hallucination. For the
patient, however, there is no barrier, and the risk becomes higher if it is not just an informational
hallucination but an uncorrected diagnostic errors that lead to unverified clinical self-action or
non-adherence. Furthermore, as a clinical chaplain, a lack of proper understanding or uncertainty
about clinical Information can significantly increase the anxiety and emotional distress of
patients and their loved ones.

The potential benefits associated with improved patient empowerment, engagement, and
access to clinical information.

The potential benefits of well-structured, layered, and clearly communicated clinical information
include easier, clearer access to clinical information for patients and the democratization of
health information.

What device characteristics, output features, or safeguards might mitigate risks that could
arise when a user lacks the domain knowledge to independently evaluate an output,
without unnecessarily underestimating patient capability?

a. Human in the loop (especially for actionable clinical Information)- HCP must review the
results first before any major health advice or treatment steps are executed directly by the patient,
because patients are more likely to accept GenAI-enabled outputs as truth if professional clinical
oversight is missing.

b. Model questions to ask physicians in case of referral - on our platform, we proposed
model questions patients can ask the HCP. Also, the GenAI-enabled AI should, based on the
patient’s input, provide a ready-to-use checklist of questions that patients can bring to their next
doctor’s visit. This also deepens engagement for the human in the loop.

c. Glossary to support medical literacy - Include a clear word list that explains tricky
medical terms so patients can easily follow along.

d. Clinical information should be expressed with a level of certainty. - Clearly show how
confident the GenAI-enabled model is in its answer—making it obvious whether an output is a
definite fact or just a possibility, and express the degree of probability.

Page 2 of 5



 e. Ordinary patient-friendly language (referral to more technical information for interested
patients) - Use everyday language or vocabulary that anyone can understand, while giving
patients the option to click or tap to read more sophisticated clinical information if they want to
know more. Regulators should use a language baseline structure built on health literacy standards
(e.g., a 6th- to 8th-grade reading level) to protect ordinary users with limited or no medical
literacy.

f. Potential hallucination alert - Display a visible warning that GenAI-enabled can make
mistakes or state things that sound real but are actually incorrect.

g. Feedback prompt to patients for GenAI-enabled self-learning and post-market
monitoring- Ask patients simple follow-up questions (like "Was this easy to understand?") so the
system can learn from real-world use and track how well it is working.

Page 3 of 5



 Section E: Discussion Question 13

13. For which clinical domains, device functions, or subpopulations is synthetic data
particularly well-suited, or particularly inadequate, as a supplement to real- world
evidence? What safeguards would mitigate the risk that synthetic data generated by models
of the same class as the device under evaluation reproduces the very performance gaps the
evaluation is intended to detect, particularly for underrepresented subgroups?

Context where synthetic data is well suited:

Rare Clinical Conditions:
1. As a clinical Chaplain, I have encountered patients with rare clinical conditions. Rare
clinical conditions, or those with insufficient real-world data, can be complemented with
synthetic data.

International & Medical Tourism Populations:
2. Patients on medical tourism in the US. According to the Global Healthcare
Accreditation3, medical tourism statistics show that about 1.9 million patients from different
parts of the world come to the US annually for medical care. Medical tourists introduce crossborder variations in baseline health diagnostics, prior treatments, genetic diversity, or linguistic/
cultural reporting differences. Synthetic data would also suit this subpopulation, since their input
in GenAI-enabled systems would be shaped by international characteristics and values that the
average American patient may not present.

Safeguards to Mitigate Synthetic data Risk:

1. Track and evaluate GenAI-enabled output against specialists’ HCP outputs.
To mitigate this risk, a GenAI-enabled system should track inputs for rare clinical conditions, or
conditions with insufficient real-world data, and evaluate whether its output aligns with what a
specialist HCP in that field would recommend. This will also be a learning curve for GenAIenabled tools.

2. Medical Tourist Subpopulation
Validate synthetic data for this group against real-world global clinical datasets to avoid
reinforcing regional clinical biases.

3. Have prompts that help to determine if the input is from a medical tourist.
To mitigate the risk that may be associated with synthetic data for this subpopulation, the GenAIenabled device should have prompts that help determine if the patient inputting is an

3 https://www.globalhealthcareaccreditation.com/

Page 4 of 5



 International patient, and if the answer is yes, it should be kept to track and use the input for
continuous learning of the patterns and characteristics associated with international patients.

4. Ongoing post-deployment monitoring of performance of synthetic data output.
This is necessary to monitor hallucinations, prevent automation bias, and any other form of bias
under Section 1557 of the ACA. This can be achieved with a systemic feedback loop for
synthetic data output and by subjecting clinical output from synthetic data to human-in-the-loop
conditions. For example, getting final approval from the patient’s HCP.

5. Patient awareness of synthetic data usage in their care.
Outputs that flow fully or partially from synthetic data should be characterized as such, so that
the patient is aware of the sources. For example, the clinical information is sourced or partially
sourced from synthetic data.

Page 5 of 5