GNS-AI (Amit Shah)

GNS-AI submits the attached comments regarding FDA’s discussion paper on generative AI-enabled medical devices.

Our comments focus on two related questions that may not be adequately addressed by aggregate device-performance measures alone. First, how should safety be assessed for a particular patient and clinical context when acceptable aggregate performance can conceal clinically important variation in risk? Second, how should FDA assess apparently low-risk or advisory AI functions when their outputs become consequential through downstream clinical reliance or participation in a broader workflow or decision chain?

We also encourage FDA to consider the operational conditions under which AI systems are evaluated, including differences between retrospective or synthetic datasets and the information actually available at the point of care.

The attached comment discusses these issues in greater detail and proposes questions for further regulatory consideration without prescribing a particular technical solution.

Public Comment on Considerations for the Regulation of Generative
AI-Enabled Medical Devices
Docket: FDA-2026-N-7874
Submitted by: Dr. Amit Shah, Ph.D., Founder and CEO, GNS-AI
Date: October 19, 2026

Patient-Level Safety and Workflow-Level Risk in Generative AI-Enabled Medical
Devices
GNS-AI appreciates the opportunity to comment on FDA's discussion paper concerning generative AIenabled medical devices.
We support FDA's consideration of competency assessment, clinical confirmation, and postmarket
performance monitoring. As generative and agentic AI systems become increasingly embedded in clinical
care, however, we encourage FDA to examine two related questions that may not be adequately addressed
by aggregate device performance alone.
First, how should safety be assessed for a particular patient and clinical context when
acceptable aggregate performance can conceal important variation in risk?
Second, how should the risk of an AI-enabled function be assessed when a seemingly limited
or low-risk use becomes consequential through downstream reliance or its role within a
broader clinical workflow or decision chain?
These questions are particularly important for generative AI because the same system may encounter
highly variable patients, information environments, clinical contexts, and patterns of use.

1. Aggregate Performance May Not Establish Safety for a Particular Patient
Aggregate measures such as overall accuracy, sensitivity, specificity, error rates, or benchmark
performance are important measures of device performance.
However, acceptable aggregate performance does not necessarily establish that a device is safe for every
patient or clinical circumstance represented within that aggregate.
Performance may vary across clinical presentations, patient characteristics, available information, disease
severity, unusual combinations of findings, or other conditions that are obscured when results are
summarized across a population.
A device could therefore satisfy an overall performance threshold while performing poorly in a smaller but
clinically important set of circumstances.
This raises an important question for competency-based evaluation:
What evidence is sufficient to establish that an AI-enabled medical device can be relied upon
for the particular patient and clinical circumstances in which it is being used?
This question is different from whether the device performs adequately on average.
We encourage FDA to consider whether competency assessment should address the limits of aggregate
performance measures and the extent to which validation evidence supports clinically meaningful variation
within the intended population.

FDA-2026-N-7874 | GNS-AI Page 1
2. The Information Available During Actual Care May Differ From the Information
Available During Validation
Patient-level performance also depends on the information available when a device is used.
Clinical data do not arrive as a complete and static record. Laboratory results, imaging findings, medication
information, clinical notes, and other data may become available at different times. Information may be
missing, delayed, inconsistent, or outdated.
Retrospective evaluation can unintentionally provide a device with a more complete view of the patient
than would have existed at the actual point of care.
Similarly, synthetic datasets may accurately reproduce clinical variables while failing to reproduce the
incompleteness, timing differences, and inconsistencies of operational healthcare data.
These differences matter because the safety of an AI-generated output may depend not only on whether the
underlying clinical facts are represented correctly, but also on which facts were actually available when the
output was produced.
We encourage FDA to consider whether clinical validation should assess the operational conditions under
which information becomes available, particularly when device outputs influence consequential clinical
decisions.

3. Apparently Low-Risk AI Functions Can Become High-Consequence Through Clinical
Reliance
Risk may also be underestimated when an AI function is evaluated only according to its immediate task.
A system that summarizes a medical record, retrieves information, prioritizes cases, drafts documentation,
or provides an advisory recommendation may appear substantially less consequential than a system that
directly orders treatment or executes a clinical action.
That distinction is important, but it may not be sufficient.
An incorrect summary can influence what a clinician notices.
A prioritization system can influence who receives attention first.
An information-retrieval system can influence which evidence enters a decision.
An advisory output can materially influence a subsequent treatment decision.
The AI function may therefore be informational in form while consequential in effect.
The presence of a clinician in the workflow does not necessarily eliminate this concern. A clinician may
retain formal decision authority while relying heavily on an AI-generated representation of the patient's
condition.
We therefore encourage FDA to consider whether risk assessment should include reasonably
foreseeable clinical reliance, rather than classifying risk primarily according to whether the AI
directly executes an action.

4. Risk May Emerge Across a Sequence of AI-Assisted Decisions
The same issue becomes more significant when multiple AI-enabled functions contribute to a clinical
workflow.
Individual components may each appear relatively benign when evaluated separately. One system may
retrieve information, another may summarize it, another may prioritize possible concerns, and another may
provide a recommendation.
FDA-2026-N-7874 | GNS-AI Page 2
Yet the final clinical decision may depend on the combined influence of all of them.
In such circumstances, the clinical consequence may not be attributable to a single obviously high-risk AI
function.
Risk can emerge from the sequence.
This is particularly relevant to agentic AI systems but is not limited to fully autonomous agents. Similar
concerns can arise whenever multiple AI-assisted steps contribute to a consequential clinical decision.
FDA should therefore consider whether evaluation at the level of an individual model or function is always
sufficient when the intended or reasonably foreseeable use involves a broader workflow.
A useful regulatory question is:
When several AI-enabled functions contribute to a clinical decision, how should the safety of
their combined influence be evaluated?

5. Postmarket Evaluation Should Consider Patient-Level and Workflow-Level Safety
Signals
These issues also have implications for postmarket monitoring.
An AI-enabled device may maintain acceptable aggregate performance while experiencing problems in
particular clinical circumstances.
Similarly, problems associated with downstream reliance or interaction among multiple functions may not
immediately appear as degradation in conventional device-level metrics.
We therefore encourage FDA to consider whether postmarket evaluation should be capable of identifying
clinically important variation that may be obscured by aggregate performance.
The objective should not be to prescribe a particular monitoring technology or architecture.
Rather, FDA should consider what evidence is needed to determine whether a device continues to be safe
and effective across the range of patients, circumstances, and workflows in which it is actually used.

6. Questions for FDA Consideration
We encourage FDA to consider the following questions as it develops its regulatory approach:
1. How should safety be assessed for an individual patient or clinical circumstance when aggregate
device performance may conceal important variation?
2. What evidence is needed to determine when validation results are applicable to the circumstances
of a particular use?
3. How should validation account for differences between complete retrospective datasets and the
information actually available at the point of care?
4. Should the risk of an AI-enabled function reflect reasonably foreseeable downstream reliance as
well as its immediate function?
5. How should FDA evaluate AI functions that appear low-risk individually but become
consequential when combined within a broader workflow or decision chain?
6. Are existing postmarket performance measures sufficient to detect clinically important patientlevel or workflow-level safety problems that do not materially affect aggregate performance?
These questions can be investigated without prescribing any particular technical solution.

FDA-2026-N-7874 | GNS-AI Page 3
Conclusion
Generative and agentic AI challenge several assumptions underlying conventional medical device
evaluation.
A device can perform adequately on average while presenting materially different risk in a particular
patient or clinical circumstance.
Likewise, an AI function that appears informational or advisory in isolation can become consequential
when clinicians or other systems rely on its output, particularly when several AI-assisted functions
contribute to the same clinical decision.
These issues suggest that medical AI safety cannot always be inferred from aggregate performance,
apparent autonomy, or evaluation of individual functions in isolation.
As FDA develops its approach to generative AI-enabled medical devices, we encourage the Agency to
examine a broader question:
How should safety be established when the clinical significance of an AI-enabled function
depends on the patient, the circumstances of use, and the broader decision process in which
its output participates?
Answering that question could help advance the regulatory science of medical AI while leaving room for
multiple approaches to demonstrating safety and effectiveness.

Respectfully submitted,
Dr. Amit Shah, Ph.D.
Founder and CEO
GNS-AI
https://gnsai.com

FDA-2026-N-7874 | GNS-AI Page 4