Ketan Dayma

See attached file(s)

I am a practicing diagnostic radiologist commenting as an end user of FDA-cleared imaging AI. The attached letter reports a 33-month post-deployment evaluation of a cleared pulmonary embolism triage algorithm across 14,736 CT pulmonary angiograms at a multi-site academic health system. Monthly sensitivity fell from roughly 60–75% to 48% while specificity rose, with no software version, algorithm, or threshold change reported by the manufacturer and no change in scanners, protocols, or patient population; vendor deployment analytics report alert volumes and user engagement, not sensitivity against an independent reference standard, and therefore could not have detected it. Three requests follow, keyed to Section VI and Appendix A: (1) require observed-performance thresholds on a fixed cadence in addition to sponsor-declared-modification triggers, since no modification was declared here (Q19, Q22, Q23); (2) give deploying institutions access to version and threshold change histories and attach a defined investigate-and-disclose obligation to site-detected performance signals, which preserves rather than diffuses manufacturer accountability (Q21); and (3) extend the calibration and clinical-deferral principle in Appendix A (S.3) to cleared diagnostic and triage devices generally — this device returned a binary negative on examinations the radiologist had called technically limited, with no capacity to signal that no determination could be made. The manuscript will be posted as a preprint shortly and I will supplement this comment with the citable link; the underlying data are available to the Agency on request.

Comment on [Docket No. FDA-2026-N-7874-0001]: Considerations for the Regulation of
Generative AI-Enabled Medical Devices — Discussion Paper and Request for Feedback

Submitted by: Ketan Dayma, MD, Radiologist, Albany Medical Center, NY]

I am a practicing radiologist and I write as an end user of FDA-cleared imaging AI, not on behalf
of any manufacturer.

My comments draw on a 33-month post-deployment evaluation of an FDA-cleared pulmonary
embolism triage algorithm across 14,736 CT pulmonary angiography examinations at a multisite academic health system. The manuscript is in preparation and will be posted as a preprint
shortly; I will supplement this comment with the citable link when it is available, and the
underlying findings are available to the Agency on request in the interim.

Three findings bear directly on the questions posed in Section VI.

1. Performance changed in deployment, and no party detected it (Questions 19, 21).
Monthly sensitivity fluctuated between roughly 60% and 75% for most of the observation period,
then fell to 48% in the final month, while specificity rose. The manufacturer reported that no
software version, algorithm, or threshold change had been deployed. Site acquisition protocols,
scanner parameters, and catchment population were stable. Whatever the mechanism, the change
was invisible through every channel available to us. Vendor-supplied deployment analytics
report alert volumes and user engagement; they do not compute sensitivity against an
independent reference standard, and therefore could not have surfaced a shift of this magnitude.
A monitoring framework whose reassessment triggers depend on sponsor-declared modifications
would not have fired here. I would urge CDRH to require observed-performance thresholds
evaluated on a fixed cadence, in addition to event-based triggers, and to specify what is expected
when performance moves while the declared configuration does not (Questions 22, 23). A prior
review by us at Dayma et al. found that more than 90% of devices cleared through the
Predetermined Change Control Plan pathway did not document a mechanism for post-market
performance monitoring.

2. Deploying institutions can detect what manufacturers do not, and this need not diffuse
accountability (Question 21). The signal above was found by a hospital, not a sponsor. I do not
propose shifting responsibility away from manufacturers. I propose that site-detected
performance signals carry a defined obligation on the manufacturer to investigate, respond, and
disclose, and that deploying institutions be given access to version and threshold change
histories, which today they are not. Without that access, a site can observe that performance
moved but cannot determine why. Independent third parties, including professional-society
registries and tools qualified through the MDDT program, could reduce duplicated effort across
sites (Question 16).

3. Devices should be able to decline a determination (Appendix A, S.3). The device we
evaluated returns a binary output for every study it processes, including examinations the
interpreting radiologist described as technically limited; it returned negative results for nearly all
of them. It has no indeterminate state and no mechanism to signal that a determination cannot be
made. A radiologist who is unaware of this behavior may reasonably infer that the algorithm
resolved what the images obscured from the human eye. CDRH has rightly identified calibration,
uncertainty communication, and clinical deferral as a safety element for generative AI devices. I
would ask the Agency to extend that principle to cleared diagnostic and triage devices generally:
a defined no-determination output, triggered by prespecified input-quality criteria, is more
informative and safer than a negative result on an examination the device cannot reliably assess.

I would add one observation on under-escalation (Questions 4, 6). Triage outputs are specialistfacing by intended use, but in practice they reach ordering clinicians. Our overall false-negative
rate was 36%, and higher in outpatient and subsegmental disease. An unflagged study presented
to a generalist who lacks the context to weigh it is a meaningful under-escalation risk, and
specialist-in-the-loop interpretation should be an explicit condition of deployments in which
triage outputs are visible outside radiology.

I appreciate the Agency’s attention to post-market evidence and would welcome the opportunity
to provide the full dataset or to participate in further discussion.

Respectfully submitted,

Ketan Dayma, MD

Radiology, Albany Medical Center