Neel N. Patel

I am a cardiovascular disease fellow at the University of Tennessee Health Science Center (Ascension Saint Thomas Hospital, Nashville, TN) and a clinician-researcher in cardiovascular outcomes and the external validation of AI in cardiovascular medicine. I submit this comment in my individual capacity; the views are my own and do not represent my training program, hospital, or university. I have no financial relationships with manufacturers of AI-enabled medical devices or developers of foundation models. My full comment, with supporting evidence and references, is attached. Key recommendations:

1. Scale evidence to the consequences of error (Q1, Q8). The two-axis framework is sound. FDA should clarify how it accounts for the severity of harm, the probability of error in the deployed population, and the time available to reverse an error. Time to irreversible harm (chest-pain triage versus statin dosing, for example) and the intended-use population (prevalence, case mix, baseline risk) should modify the consequences axis.

2. Report under- and over-escalation separately (Q6). Guideline-defined thresholds, such as the 30-day risk of death or major adverse cardiac events (MACE) below 1% that defines low-risk chest pain in the 2021 AHA/ACC guideline, can serve as clinically grounded references where a device’s intended use and population align with that pathway, but not as universal criteria. Sponsors should report event rates among patients advised that urgent care is unnecessary, with upper confidence bounds and subgroup results.

3. Define authorized scope by task, autonomy level, user, and setting (Q7-8). As in graduated clinical training, competence is shown task by task. This need not mean a separate authorization for each risk category; evidence should be reusable across closely related tasks with scientific justification.

4. Treat benchmarks as measures of defined tasks, not proof of clinical safety (Q9-10). Evaluation should be independent and contamination-resistant, and benchmark results should be compared with clinical-confirmation results where feasible. Safety-critical failures should be reported separately from aggregate scores. Because zero failures observed in n test cases still leaves a 95% upper bound of roughly 3/n, sponsors should report upper confidence bounds on residual catastrophic-failure rates.

5. Use synthetic data for robustness and stress testing, but not as the sole evidence of subgroup safety in real patients (Q13). Generators can share the blind spots of the device being evaluated.

6. Measure access benefits separately from clinical safety (Q14-15). Comparing a device with delayed specialist review is appropriate for measuring access, but safety-critical errors should meet a non-inferiority margin against the standard of care, so that underserved settings do not become the places where the least-proven devices are deployed.

7. Reserve reduced premarket evidence for functions whose errors can be detected and reversed before harm, judged by what the function does and how it can fail (Q18).

8. Make real-world outcomes attributable (Q21). FDA should work toward a standardized, auditable, privacy-protective exposure record capturing the device UDI and version, the foundation-model version, material configuration changes, and, where available, whether an output was acted on.

9. Apply risk-tiered change control to every material component, including the foundation model, prompts, retrieval sources, and tools, with clinical re-confirmation reserved for changes that could alter safety or effectiveness (Q24).

10. For agentic devices, test complete multi-step trajectories and show that human oversight works under realistic conditions, including whether reviewers detect and reject unsafe actions (Q26).

Thank you for the opportunity to comment.

Neel N. Patel, MD

Re: Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion
Paper and Request for Feedback (CDRH Digital Health Center of Excellence, August 2026)

I am a cardiovascular disease fellow at the University of Tennessee Health Science Center,
training at Ascension Saint Thomas Hospital in Nashville, Tennessee. My research focuses on
cardiovascular outcomes using large national databases and meta-analysis, and I have more than
60 peer-reviewed publications. I am the sole author of a peer-reviewed article on bias, external
validation, and real-world implementation of artificial intelligence models in cardiovascular
medicine [1]. I submit this comment in my individual capacity; the views expressed are my own
and do not represent my training program, hospital, or university.

I support CDRH's risk-based, total product life cycle approach. My aim is to help FDA distinguish
acceptable from unacceptable risk in ways that are scientifically defensible and proportionate,
avoiding both unsafe deployment and unnecessary duplication of evidence. Several of the
discussion paper's illustrative examples come from cardiovascular medicine, so I draw my
examples from that field; the principles are meant to apply more broadly, although other
specialties will have their own failure modes. I address Questions 1, 6, 7–10, 13–15, 18, 21, 24,
and 26.

The discussion paper states that it does not address whether the approaches it describes fall
within FDA's existing legal authorities. I therefore present several recommendations as
regulatory objectives. Some may be achievable through existing authorities, guidance, or
recognized standards; others may depend on collaboration with health systems, standards
bodies, and model developers.

Summary of recommendations:

Three recommendations matter most:

1.​ Scale evidence to the consequences of error, including the time available to reverse
an error and the population in which the device is deployed (Q1, Q8).
2.​ Evaluate what the device does in its intended setting, not aggregate benchmark
scores alone (Q6, Q9–10, Q13–15).
3.​ Make real-world outcomes attributable to a specific device configuration, so that
postmarket monitoring can work (Q21, Q24).

Supporting recommendations:

●​ Use guideline-defined residual-risk thresholds as clinically grounded references, not
universal criteria, for triage functions (Q6).
●​ Define each device's authorized scope by task, autonomy level, user, and setting, and
allow justified reuse of evidence across related scopes (Q7–8).
●​ Report catastrophic failure modes separately, with statistically justified upper bounds on
residual risk (Q9–10).
●​ Use synthetic data for stress testing, not as stand-alone evidence of clinical subgroup
performance (Q13).
 ●​ Measure access benefits separately from clinical safety, so that gains in access cannot
offset unacceptable error rates (Q14–15).
●​ Apply risk-tiered reassessment to changes in any material component, not only the
model version (Q24).
●​ For agentic devices, test complete trajectories and show that oversight controls actually
work (Q26).

Assessment of risk (Questions 1 and 6)

Question 1. The two-axis framework is a sound organizing heuristic, and I would not add axes. I
recommend that FDA clarify how three distinct quantities relate to it: the severity of harm if an
error occurs, the probability of that error under real deployment conditions, and the time
available to detect and correct it. Two modifiers of the consequences axis would capture most of
these differences in cardiology.

Time to irreversible harm. An incorrect output about statin intensity can usually be corrected at
the next visit. An incorrect output about whether chest pain needs emergency evaluation may
not be correctable, because myocardial injury from acute coronary occlusion is time-dependent.
Sponsors could state, for each function, the expected window within which an error can be
detected and reversed; a short window should move the function upward on the consequences
axis.

Intended-use population. Sensitivity and specificity alone do not determine the risk of a
deployment. Prevalence, case mix, and baseline risk determine predictive values and the
absolute number of missed events. A chest-pain function offered directly to consumers and the
same function used in a rural emergency department without on-site cardiology serve different
populations, so the same accuracy can produce different absolute harm. I recommend that
sponsors characterize the intended population; report predictive values, calibration, and
expected absolute errors per 1,000 encounters; and reassess when the deployed population
differs materially from the validated one.

Question 6. Under- and over-escalation are not commensurable and should never be
summarized by a single accuracy figure. Cardiology offers a concrete reference point. The 2021
AHA/ACC chest pain guideline designates patients with acute chest pain and a 30-day risk of
death or major adverse cardiac events (MACE) below 1% as low risk, for whom discharge
without admission or urgent cardiac testing is reasonable [2]. Where a device's intended use,
population, available clinical information, and outcome definitions are sufficiently aligned with
that pathway, such a threshold could serve as a clinically grounded reference for prespecified
acceptance criteria. It should inform, not automatically determine, device-specific criteria: the
guideline's low-risk designation rests on structured clinical evaluation, including
electrocardiography and troponin testing, which a patient-facing conversational tool may not
have.

For triage and escalation functions, I recommend that sponsors:

●​ report the observed 30-day event rate among patients advised that urgent evaluation
was unnecessary, with its upper confidence bound and relevant subgroup analyses,
 because a small cohort can show an event rate below 1% while its upper bound remains
well above it;
●​ report that rate separately from the false-negative rate of the triage function as a whole;
●​ report over-escalation as additional emergency visits per 1,000 users over a defined
period, because its harms (crowding, cascade testing, cost to patients, and erosion of
trust) accrue at the population level;
●​ consider decision-analytic methods such as decision curve analysis, which make the
implied trade-off explicit at a prespecified threshold probability [3]. Such analyses
clarify trade-offs under stated assumptions; they do not replace clinical validation.

Outputs should also communicate uncertainty and escalation instructions in terms the intended
user can act on. Labeling should not imply that an AI assessment substitutes for emergency
evaluation unless the device has been validated for that use.

Premarket evaluation (Questions 7–10 and 13–15)

Questions 7 and 8: define scope by task. I support the competency-based approach. Graduate
medical education offers one transferable lesson: competence is judged task by task, not
globally. Under the entrustable professional activity model [4], now adopted in health
professions education in many countries, a cardiology fellow may be entrusted to perform and
provisionally interpret transthoracic echocardiograms under indirect supervision while still
requiring direct supervision for transesophageal studies. Entrustment is granted per activity
and per level of supervision, and it is revisited.

The analogy has limits. Physicians practice within a licensing and professional-accountability
system that is legally distinct from device authorization, and I offer the analogy only to illustrate
graded autonomy and reassessment. Applied to devices, it suggests that authorization and
labeling should define a device's scope as a specific task, at a specific level of autonomy, for a
specific user and setting. This does not mean that each cell of the two-axis framework needs a
separate authorization. The position of the authorized scope on the framework would set the
breadth of benchmarking and the rigor of clinical confirmation required (Question 8), and
sponsors should be able to reuse evidence across closely related tasks with a scientific
justification. For example, a device could be authorized to draft guideline-based
secondary-prevention summaries for clinicians, while its labeling and scope-maintenance
testing (S.2) make clear that it is not authorized for patient-facing triage.

Board examinations are necessary but not sufficient: physicians may not practice independently
on the basis of examination scores alone, and supervised graduate training is also required.
Benchmarks should play a similar role, as a necessary gate rather than as evidence of safe
real-world performance.

Questions 9 and 10: what benchmarks can and cannot show. Benchmark results measure
performance on defined tasks; they are not direct evidence of clinical safety or effectiveness. AI
models can lose substantial performance outside the populations and settings in which they
were developed [1]. A systematic review of 486 AI models for cardiovascular disease prediction
found that none had undergone independent external validation [5]. In an independent external
validation of a widely deployed proprietary sepsis prediction model, the area under the curve
was 0.63, compared with 0.76–0.83 previously reported by the developer and a partner health
system; the model missed 67% of sepsis cases while alerting on 18% of hospitalized patients
[6]. These examples involve conventional prediction models rather than generative AI, but they
show that performance claimed during development cannot be assumed to hold elsewhere. I
recommend that:

●​ sponsors show that benchmark domains are relevant to the intended use and represent
clinically important inputs, populations, and failure modes, and, where feasible, compare
benchmark results with clinical-confirmation results to show how well the benchmark
predicts performance in the intended setting;
●​ FDA encourage independent, contamination-resistant evaluation. Temporally
sequestered test items, written after the foundation model's training-data cutoff and not
publicly released, are one useful safeguard, but they do not by themselves ensure that a
test set is representative;
●​ sponsor-developed benchmarks be permitted and disclosed in full to FDA, without a
requirement for public release that could itself cause contamination, and be
accompanied by results on at least one independent test set;
●​ safety-critical failures (S.1–S.3) be reported separately from aggregate scores, with
failure categories, scoring rules, and acceptance criteria prespecified.

A finite test cannot prove zero risk. If a device has a true 1% rate of a critical failure, there is
about a 37% chance that 100 independent test cases will show no such failure. When no events
are observed in n independent cases, the upper bound of the 95% confidence interval is
approximately 3/n [7]. For defined catastrophic failure modes, sponsors should show that the
device does not exhibit the failure in designated challenge scenarios, and should report an upper
confidence bound on the residual failure rate from representative testing, with the required
precision scaled to the severity and reversibility of harm.

Question 13: synthetic data. The evidence synthetic data can provide depends on how they are
used.

Use of synthetic data Appropriate role

Robustness testing (paraphrase, health-literacy variation, Well suited
measurements given without units)

Generating rare or difficult cases for stress testing Useful, with independent
clinical review

Establishing subgroup safety or effectiveness in real patients Not sufficient on its own
Synthetic-data generators can reproduce the omissions and biases of their source data, and a
generator of the same class as the device under evaluation may share its blind spots. Using a
different model family reduces this concern but does not eliminate it. Myocardial infarction
without chest pain shows why this matters. In a U.S. registry of 434,877 patients with confirmed
myocardial infarction, 33% presented without chest pain; these patients were older and more
often women, were treated less aggressively, and had higher in-hospital mortality (23.3% vs
9.3%) [8]. Whether such presentations are adequately represented in synthetic data is an
empirical question that sponsors should address rather than assume. I recommend that
synthetic subgroup cases be reviewed for clinical plausibility by independent clinicians, that
results be reported separately for real and synthetic inputs, and that claims about real-world
subgroup performance rest on real or appropriately collected clinical data.

Questions 14 and 15: comparators. The comparator should match the intended user and
setting. A device for primary-care clinicians managing hypertension should be compared with
guideline-concordant primary care, not with a cardiologist panel; a device presented as offering
specialist-level interpretation should be held to specialist performance. For clinician-facing
tools, human–AI team performance should be the primary basis of evaluation, because that is
what patients experience, and because automation bias (the tendency to over-rely on automated
advice) can only be measured when clinicians use the device [9].

I support the counterfactual comparator in Question 15, but benefit and safety should be
measured separately. I intend to practice general cardiology in medically underserved
communities, where patients often wait a long time for specialist evaluation; for them, the actual
alternative to a device may be delayed specialist review. Measuring access and timeliness against
that alternative is appropriate. Access benefits must not, however, obscure clinically
unacceptable error rates, or underserved settings could become the places where the
least-proven devices are deployed. I recommend that sponsors prospectively specify the
population, comparator, endpoints, and margins for four separate domains: clinical safety and
effectiveness against the relevant standard of care, with a non-inferiority margin for
safety-critical errors; access and time to care against the actual alternative; resource use; and
human–AI interaction.

Postmarket monitoring (Questions 18, 21, and 24)

Question 18. Greater reliance on postmarket monitoring is reasonable when errors are likely to
be detected and reversed before harm occurs, and when a monitoring program with
prespecified performance thresholds and stopping rules is operating at launch. It is least
appropriate where residual risk is high, and harm can occur before postmarket signals
accumulate, as with autonomous action-taking functions or under-triage of time-critical
conditions such as chest pain or stroke symptoms. The deciding factor should be what the
function does and how it can fail, not its audience or condition alone. A narrowly designed tool
that reliably directs a patient with possible stroke symptoms to emergency services differs from
a conversational tool that may reassure the same patient that the symptoms are benign.

Question 21: attributing outcomes to a device. The main practical barrier to postmarket
monitoring is attribution. In research using large national databases, outcomes can be linked to
procedures and device categories through procedure codes, and to medications through
pharmacy claims. By contrast, there is no routine way to know whether a GenAI tool, let alone
which configuration of it, was used in a given clinical decision. Passive adverse-event reporting,
such as the MAUDE database, may also miss these errors, because clinicians often cannot tell
when an AI output contributed to harm.

I recommend that FDA work toward a standardized, interoperable, and auditable exposure
record that captures, where technically feasible:

●​ the device's unique device identifier (UDI) and software version (for downloaded
stand-alone software, the UDI's production identifier already conveys the version
number under 21 CFR 801.50);
●​ the underlying foundation-model identifier and version, which can change
independently of the device version;
●​ material configuration changes, such as prompts, retrieval sources, or tool permissions;
●​ the time of use and the clinical task;
●​ where available, whether an output was displayed, accepted, modified, or acted on.

Recording use is not proof of influence, and exposure data alone cannot establish causation; they
support signal detection, risk adjustment, and well-designed follow-up studies. The record
should be privacy-protective and limited to what is needed to reconstruct exposure, rather than
retaining every prompt or conversation indefinitely. Clinical registries maintained by
professional societies, including cardiovascular registries, could add these data elements and
serve as independent surveillance infrastructure. Health systems and societies would generate
and share data, while manufacturers remain accountable for their applicable postmarket
obligations.

Question 24: change control. Sponsors should maintain a documented change-control plan
covering every component that could materially affect device behavior: the foundation model,
device software, prompts, retrieval sources, tools, and safety layers. Changes should trigger
predefined, risk-tiered reassessment. Re-benchmarking can be scaled to the change, and clinical
re-confirmation should be reserved for changes that could materially alter safety or
effectiveness. Manufacturers should have a documented way to learn of upstream model
changes in time and, where technically feasible, to run or revert to a previously validated version
while reassessment is completed. Because behavior can change without a visible version change,
periodic re-benchmarking and drift monitoring remain necessary.

Agentic devices (Question 26)

Agentic devices should be evaluated by level of autonomy, with evidence specific to each level at
which a device is permitted to operate. A device that recommends an anticoagulant order, one
that drafts the order for a clinician's signature, and one that places it without further
authorization present different risks. Per-step accuracy can also overstate end-to-end reliability.
As an illustration that assumes independent errors, a system that is 95% reliable at each of ten
steps would complete the whole sequence correctly only about 60% of the time. In practice,
errors may be correlated or caught at later steps, so end-to-end reliability should be measured
rather than calculated. I recommend:
 ●​ acceptance criteria based on complete multi-step trajectories, including tests of whether
the system detects and recovers from its own errors and from erroneous tool outputs;
●​ for actions with severe or hard-to-reverse consequences, authorization controls shown
to be effective under realistic conditions. Where a human confirmation step is used, it
should present enough clinical context for informed review, and testing should measure
whether reviewers detect and reject unsafe actions, because workload and time pressure
promote over-reliance on automated advice [9];
●​ audit logs of each step, tool call, and retrieved source, sufficient to reconstruct any action
after the fact.

Closing

Thank you for the opportunity to comment. I would welcome the chance to contribute clinical
expertise to future CDRH work on GenAI-enabled devices, particularly in cardiovascular
medicine.

Respectfully submitted,

Neel N. Patel, MD

Cardiovascular Disease Fellow, University of Tennessee Health Science Center

Ascension Saint Thomas Hospital, Nashville, Tennessee

neelnavinkumarpatel@gmail.com

Disclosure: I have no financial relationships with manufacturers of AI-enabled medical devices
or developers of foundation models.
References

1.​ Patel NN. Bias, external validation, and real-world implementation of artificial
intelligence models in cardiovascular medicine. Am J Cardiol. 2026.
doi:10.1016/j.amjcard.2026.06.005
2.​ Gulati M, Levy PD, Mukherjee D, et al. 2021 AHA/ACC/ASE/CHEST/SAEM/SCCT/SCMR
guideline for the evaluation and diagnosis of chest pain: a report of the American College
of Cardiology/American Heart Association Joint Committee on Clinical Practice
Guidelines. J Am Coll Cardiol. 2021;78(22):e187–e285. doi:10.1016/j.jacc.2021.07.053
3.​ Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction
models. Med Decis Making. 2006;26(6):565–574.
4.​ ten Cate O. Entrustability of professional activities and competency-based training. Med
Educ. 2005;39(12):1176–1177.
5.​ Cai Y, Cai YQ, Tang LY, et al. Artificial intelligence in the risk prediction models of
cardiovascular disease and development of an independent validation screening tool: a
systematic review. BMC Med. 2024;22:56. doi:10.1186/s12916-024-03273-7
6.​ Wong A, Otles E, Donnelly JP, et al. External validation of a widely implemented
proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med.
2021;181(8):1065–1070. doi:10.1001/jamainternmed.2021.2626
7.​ Hanley JA, Lippman-Hand A. If nothing goes wrong, is everything all right? Interpreting
zero numerators. JAMA. 1983;249(13):1743–1745.
doi:10.1001/jama.1983.03330370053031
8.​ Canto JG, Shlipak MG, Rogers WJ, et al. Prevalence, clinical characteristics, and mortality
among patients with myocardial infarction presenting without chest pain. JAMA.
2000;283(24):3223–3229. doi:10.1001/jama.283.24.3223
9.​ Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency,
effect mediators, and mitigators. J Am Med Inform Assoc. 2012;19(1):121–127.
doi:10.1136/amiajnl-2011-000089