DeepHealth, Inc.

Thank you for the opportunity to comment. Please see the attached PDF file.

October 19, 2026

Dockets Management Staff (HFA-305)
Food and Drug Administration
5630 Fishers Lane, Rm. 1061
Rockville, MD 20852

Re: Docket No. FDA-2026-N-7874 — Considerations for the Regulation of Generative AIEnabled Medical Devices: Discussion Paper and Request for Feedback

Dear Dr. Tarver and Colleagues at the Center for Devices and Radiological Health (CDRH):

As a medical technology developer and clinical research organization focused on artificial
intelligence in diagnostic imaging, cancer screening, and clinical workflow orchestration—whose
FDA-cleared technologies and clinical networks support the interpretation of millions of imaging
examinations annually across the United States—DeepHealth is deeply committed to advancing
rigorous clinical science that measurably improves patient outcomes.

In reviewing the Discussion Paper and the public comments submitted to this docket to date, we
believe CDRH faces a dual imperative that cannot be solved by any single, uniform regulatory
rubric:

• Where net patient benefit is uncertain—or where generative outputs risk inducing
clinician automation bias—regulators must remain rigorously skeptical. In diagnostic
radiology, the medical community has already lived through the cautionary experience of early2000s first-generation mammography computer-aided detection (CADe) devices. Although
cleared on the premise that a "human in the loop" would simply ignore false marks, poor
underlying standalone specificity combined with human automation bias ultimately degraded
average reader accuracy in community practice. Generative AI—which produces fluent,
authoritative-sounding prose rather than simple bounding boxes—carries an even greater
potential to "dumb down" clinical interpretation if poorly validated systems are allowed to anchor
human judgment.
• Conversely, where rigorous, large-scale scientific evidence demonstrates that an AIenabled workflow produces a major net improvement in real-world patient health—such
as elevating generalist physicians to the performance level of fellowship-trained
specialists—FDA pathways should actively facilitate rapid patient access. Regulatory
frameworks that evaluate devices in a vacuum against idealized academic comparators, while
ignoring the stark access and expertise gaps in routine U.S. community care, risk withholding
life-saving capabilities from the patients who need them most.

To help CDRH operationalize this balance, we focus our comments on two areas where we see the
greatest risk of conceptual confusion—and the greatest opportunity for constructive impact: (I) the
structural flaw in applying a physician "board examination / presumption of competence" paradigm
to Generative AI (Questions 7, 8, 9, and 11), and (II) the distinct technical and human-factors
realities of Vision-Language Models (VLMs) and generalist-versus-specialist workflows in medical
imaging (Questions 17, 4, 14, and 15).
I. The Fallacy of the "Oral Boards" Analogy and the Limits of Presumed Competence
Response to Questions 7, 8, 9, and 11 (Competency-Based Approach, Evidence Levels,
Benchmarking, and Clinical Confirmation):
In Section V of the Discussion Paper, CDRH introduces a proposed "competency-based approach"
to premarket evaluation, explicitly drawing an analogy to how human physicians are credentialed:
because physicians cannot be tested on every clinical scenario they will encounter, they are
evaluated through structured knowledge examinations (such as licensing and specialty board
certifications) paired with supervised practice. In public discourse around this docket, several
academic and clinical commentators (for example, in radiology, proponents of analogy to the
American Board of Radiology certifying/oral examination) have argued that if a generative model
passes a rigorous, board-style sampling examination covering a fraction of the domain, regulators
can infer a broader "presumption of competence" across the remainder of the clinical field.

This analogy is fundamentally flawed because it mistakes the final examination for the credentialing
system.
• Not just anyone—and certainly no unvetted software artifact—is permitted to sit for a
specialty board examination. Before a physician is ever allowed to approach the certifying
examination in radiology or any other specialty, that individual must complete four years of
undergraduate medical school, a clinical internship, and four years of postgraduate residency
training (often followed by a subspecialty fellowship). Throughout those eight-plus years and
tens of thousands of clinical hours, the trainee is continuously observed across diverse care
settings by dozens of independent faculty physicians.
• Crucially, this longitudinal human training pipeline acts as an intensive behavioral and
reliability filter: If a resident is found to be dishonest—for example, fabricating a physical exam
finding or inventing a clinical history that does not exist in the chart—they are dismissed long
before they can apply for the boards. If a trainee lacks metacognitive calibration—failing to
recognize when a case is outside their expertise or failing to call for specialist backup—they are
remediated or prevented from advancing.
• The certifying examination is enabled to sample only a small fraction of medical
knowledge precisely because eight years of longitudinal observation have already
established that the candidate has successfully undergone extensive rigorous training,
possesses a shared human biological and cognitive architecture, understands basic
physical and causal reality, and adheres to professional norms of truthfulness and selfawareness.

Large Language Models (LLMs) and Vision-Language Models (VLMs) possess none of these preexamination guardrails. A generative model has not undergone residency; it has no intrinsic
concept of truth versus confabulation ("hallucination"); and its statistical optimization on next-token
prediction means that high accuracy on common textbook vignettes provides zero assurance
against catastrophic, confident fabrication on a slightly out-of-distribution clinical presentation.

Recommendations for CDRH (Questions 7, 8, 9, and 11):
1. Reject "Competency Inference" Across Untested Clinical Domains (Q7, Q9): Passing a
non-clinical medical benchmark (such as USMLE-style multiple-choice questions or sampled
 board vignettes) should never confer a regulatory "presumption of competence" across a broad
intended use. At most, non-clinical benchmarking is a necessary screening floor (analogous to
pre-clinical coursework) to detect gross failure modes before clinical validation begins.
2. Require Explicit Architectural Bounding and Validated Deferral (Q7, Q8): Rather than
inferring competence on rare or complex conditions from performance on common ones, CDRH
should require sponsors to bound the input and output space of the device and explicitly
validate calibrated abstention/deferral behavior—proving statistically that the system recognizes
out-of-distribution or ambiguous inputs and routes them to appropriate human specialist review.
3. Substitute Longitudinal Clinical Confirmation for Assumed Generalization (Q8, Q11): Until
generative architectures possess verifiable longitudinal track records, the "residency" equivalent
for a medical AI device must be rigorous premarket clinical confirmation on sequestered, multisite real-world clinical cohorts—and, where appropriate, prospective postmarket registry studies
conducted by sponsors committed to peer-reviewed clinical science.

II. Discriminative vs. Generative Architectures, Verification Friction, and Postmarket
Surveillance
Response to Questions 1, 2, 3, 4, 12, 17, 18, 19, 20, 21, 22, 23, 24, and 25:

A large fraction—indeed, the vast majority—of the more than 1,000 AI/ML-enabled medical devices
cleared by CDRH to date in radiology are based on discriminative computer vision architectures,
such as Convolutional Neural Networks (CNNs) and task-specific vision transformers. Yet in
industry discussions and across several public comments in this docket, there is a growing
tendency to group all modern "foundation models" and Vision-Language Models (VLMs) into a
single continuum with established imaging AI.

CDRH’s framework should draw a sharp architectural and evidentiary boundary between
discriminative pixel-recognition models (CNNs) and generative Vision-Language Models (VLMs):

4. Bounded Discriminative Recognition (CNNs) vs. Unconstrained Autoregressive
Generation (VLMs): A CNN-based CADe/CADx/CADt device trained on thousands of biopsyproven breast cancers on mammograms, or hundreds of lung nodules on CT scans is not
generating text. It is matching pixel patterns to rigorous reference standards on a lesion-bylesion, problem-by-problem basis within a mathematically bounded output space. In such
architectures, open-ended textual hallucinations are structurally impossible because the model
is recognizing and localizing defined signals, not generating unconstrained language.
5. The Seductive Hazard of Pre-Trained VLMs: The allure of multimodal foundation models and
VLMs is that broad pre-training can yield superficially impressive zero- or few-shot performance
without the arduous, lesion-by-lesion curation required for task-specific CNNs—and because
VLMs pair a vision encoder with an LLM backbone, they can rapidly produce fluent draft clinical
reports. However, because VLMs inherit the unconstrained-by-design autoregressive generation
of their LLM heritage, they can—and in our direct testing, more than occasionally do—
hallucinate wildly. A VLM may fabricate findings that do not exist on the image, invent
comparisons to non-existent prior exams, or confidently state "normal" while overlooking a
subtle, life-threatening lesion because its language prior overrides a weak visual signal.
6. Regulatory Implications for Premarket Review (Questions 7, 9, and 17): First, CDRH must
ensure there is no "fluency shortcut" around lesion-level rigor: sponsors of VLM-based devices
cannot use broad foundation-model pre-training or plausible report generation to bypass the
lesion-by-lesion, pathology-by-pathology validation rigor required of CNNs. If a VLM’s intended
use encompasses drafting diagnostic findings on imaging studies, it must demonstrate
sensitivity and specificity on specific critical pathologies comparable to validated discriminative
detectors. Second, CDRH should distinguish between (a) end-to-end unconstrained VLMs
(where raw pixels are mapped directly into open-ended text tokens by a generative model) and
(b) modular/hybrid architectures in which validated, bounded CNN/discriminative models
perform lesion detection, segmentation, and quantification, and a constrained language layer
merely formats those verified structured outputs into human-readable report text. The latter
preserves the hallucination-resistant rigor of CNNs while delivering workflow efficiency, and
should be recognized as carrying substantially lower risk than unconstrained end-to-end VLM
generation.

The Discussion Paper’s two-axis risk framework (Clinical Significance × Autonomy) treats most
"clinician-in-the-loop" assistive outputs as occupying a uniform autonomy tier. In real-world clinical
practice, however, a human-in-the-loop safeguard is only as protective as the Verification Friction
involved in checking the AI’s output against primary ground truth:

• Low Verification Friction: Consider an assistive tool that flags a suspected phalangeal fracture
on a 3-view plain radiograph of the hand. A clinician can glance at the single plain film and verify
or reject the finding in a few seconds.
• High Verification Friction: Contrast that with a VLM that drafts a complete diagnostic report for
a multi-phase abdominal and pelvic CT scan performed with and without intravenous contrast,
including comparative statements against multiple prior examinations. Verifying every
affirmative and negative assertion in that draft report requires scrolling through potentially
hundreds of axial and multiplanar slices across multiple contrast phases and historical studies,
cross-checking organ by organ.
• Weighting Verification Friction in Question 1: Treating both devices as equivalent "clinicianreviewed assistive tools" runs the risk of overlooking clinical reality. Where Verification Friction
is high, time-pressured clinicians are acutely vulnerable to automation bias and "omission
blindness" (failing to notice what a fluent draft report left out). CDRH should explicitly weight
Verification Friction—the cognitive time, number of data elements, and interface effort required
to verify an output against primary data—as a core risk modifier, and require spatial/source
linking (e.g., clicking a drafted sentence jumps directly to the relevant slice and prior comparison
ROI) to reduce friction in complex exams.

Expanding "User Verification Capacity" to Include Motivation and Bandwidth (Questions 3
and 4): Many commenters in the docket treat User Verification Capacity purely as a function of
medical credentials—assuming that Healthcare Professionals (HCPs) inherently have high capacity
to catch AI errors while patients have none. We urge CDRH to adopt a more realistic model of User
Verification Capacity that accounts for both domain expertise AND cognitive bandwidth/personal
motivation:
• The Overwhelmed Specialist vs. the Highly Motivated Patient (Question 3): A busy
radiologist overwhelmed with a massive daily worklist has deep medical expertise, but may
have severely constrained time and attentional bandwidth ("capacity") to scrutinize a subtle AI
error on any single routine-appearing case. Conversely, the woman whose mammogram is
being evaluated—and who is anxiously wondering whether she has breast cancer—possesses
extraordinary personal motivation and ample time to scrutinize her results, ask pointed follow-up
questions, and seek clarification, even if she lacks formal radiological training.
• Generalist vs. Specialist Users (Question 4): Similarly, a generalist clinician receiving a
subspecialty-level VLM synthesis without the training or time to inspect the raw imaging study
has neither the specialist expertise of the radiologist nor the singular focus of the patient. Rather
than assuming HCP-facing tools are inherently safe from undetected hallucinations or that
patient-facing tools are uniformly perilous, for a given use case CDRH should evaluate (1) the
user's expertise relative to the task, (2) the user's real-world time/attentional bandwidth and
motivation in the intended workflow, and (3) whether errors are reversible before harm occurs.

Because open-ended VLMs can potentially hallucinate any sort of clinical finding across any organ
system, they require rigorous premarket validation—more than simply relying on postmarket
observation (Question 18). However, the form of that premarket validation must be scientifically
matched to how generative models fail and how clinicians actually work:

7. Mandatory "Injected-Error Rejection" Testing for Human-in-the-Loop Claims (Questions
11 and 14): Whenever a sponsor claims that clinician review mitigates the risk of VLM
hallucinations or omissions, premarket (and postmarket) validation must empirically prove that
users consistently detect and reject injected errors. Specifically, human-factors and clinical
confirmation studies should seed realistic, plausible VLM hallucinations and critical omissions
into the workflow under realistic reading speeds. If reviewing clinicians fail to consistently reject
injected errors on high-friction exams, the device cannot rely on "human-in-the-loop" labeling to
downgrade its premarket evidence requirements.
8. Why Traditional Laboratory MRMC Reader Studies Fail for Generative AI (Questions 11,
14, and 15): While premarket rigor is essential, CDRH should not default to traditional
laboratory Multi-Reader Multi-Case (MRMC) reader studies as the mandatory premarket clinical
confirmation design for VLMs. The peer-reviewed literature is clear that laboratory MRMC
studies suffer from fundamental external-validity flaws. To achieve statistical power with a small
sample (N = 200–300 cases), MRMC studies require heavy artificial enrichment of positive
cases. Reading a cohort that is 30–50% positive in a laboratory test environment alters reader
behavior ("laboratory vigilance" and prevalence-induced operating-point shifts) in ways that bear
little resemblance to real-world screening or diagnostic environments where true disease
prevalence may be under 1%. Furthermore, a 250-case enriched laboratory study is far too
small to bound an open-ended VLM that can hallucinate across hundreds of distinct clinical
entities, or to measure device performance on rare but critical findings.
9. Real-World Testing Paired with Structured Clinical Taxonomies (Questions 11, 12, and
15): Instead of small, artificially enriched laboratory MRMC studies, CDRH should prioritize
large-scale real-world clinical testing conducted in representative clinical environments with
natural disease prevalence, evaluated against realistic workflow counterfactuals (Question 15).
 To make large-scale evaluation statistically rigorous (Question 12), free-text VLM outputs might
usefully be mapped into a Structured Clinical Taxonomy (decomposing narrative text into
discrete, verifiable clinical assertions: finding type, anatomical location, laterality, size/severity,
temporal change, and clinical recommendation), enabling calculation of assertion-level
sensitivity, specificity, and hallucination rates across thousands of real-world cases.

In Question 19, the Discussion Paper proposes periodic re-benchmarking and sample-based
clinician review. While useful, manual small-sample review cannot achieve the statistical scale
needed to monitor open-ended VLMs in clinical practice—particularly for findings that occur only
rarely in the population and yet are clinically critical (e.g., subtle arterial dissections, early interval
cancers, or uncommon postoperative complications).

In diagnostic imaging and clinical reporting workflows, however, every clinical encounter produces a
natural, high-value ground-truth pairing: the VLM’s initial draft output versus the attending
radiologist’s final signed report. We recommend consideration of a Taxonomy-Driven Continuous
Postmarket Telemetry framework:

10. Map Both the AI Draft and the Final Signed Report to a Structured Clinical Taxonomy
(Question 19): Rather than relying on crude lexical edit-distance (which conflates stylistic
rephrasing with clinical errors) or small manual chart audits, sponsors could productively
measure and monitor performance by mapping both the VLM draft and the radiologist’s final
signed report into a standardized Structured Clinical Taxonomy. By comparing the taxonomy
nodes present in the AI draft against those in the final signed report across 100% of clinical
volume, the system automatically quantifies: (a) Taxonomy Additions by the Radiologist
(identifying false negatives / omissions by the VLM, stratified by specific pathology and
severity); (b) Taxonomy Deletions/Reversals by the Radiologist (identifying VLM hallucinations /
false positives rejected by the physician); and (c) Injected-Error / Canary Vigilance (verifying
that reviewing clinicians continue to actively edit and reject discordances rather than
succumbing to postmarket automation drift).
11. Powering Surveillance for Rare, Critical Findings: Because a taxonomy-based pipeline can
run continuously across hundreds of thousands or millions of real-world exams across diverse
sites, scanners, and patient demographics, it provides the large-scale denominators required to
detect performance degradation on rare but high-consequence findings within days or weeks—
something a periodic 200-case manual audit could never achieve.
12. Independent Supervisory Extraction Agents (Question 20): Where an automated NLP/AI
parser is used to map free-text reports into the Structured Clinical Taxonomy for postmarket
monitoring, that supervisory parser should be architecturally independent of the generative VLM
being monitored and deterministically validated against human-adjudicated taxonomy labels so
that the monitor does not share the primary device’s blind spots.

• Taxonomy-Bounded Premarket Verification Envelope (Questions 22 and 23): Before any
model weight, prompt, or retrieval update is deployed under a PCCP, the candidate system
must pass a locked, sequestered multi-site regression battery (enriched specifically with rare
critical findings and adversarial hallucination traps) and demonstrate non-inferiority across every
critical branch of the Structured Clinical Taxonomy, plus zero regression on predefined
 catastrophic "never-events." Once deployed, continuous taxonomy-based postmarket telemetry
monitors the new version against predefined statistical control limits, triggering automatic
version rollback if real-world clinician override/deletion rates spike in any taxonomy category.
• Third-Party Foundation Model Changes and Master Files (Questions 24 and 25): Finally,
we support Device Master Files (MAFs) for Foundation Models (Question 25). Because the
medical device manufacturer retains ultimate regulatory accountability (Question 21), any
foundation model developer submitting a Master File would have greater credibility if they could
guarantee (1) cryptographically pinned, immutable model weights and tokenizers for regulated
device customers (with zero silent updates), (2) a minimum 180-day advance notice prior to
endpoint deprecation, and (3) full disclosure of training-data provenance and benchmark
overlap (Question 10) so downstream sponsors can ensure premarket test sets remain truly
uncontaminated.

Conclusion
Summary of Recommendations
By (1) rejecting false analogies of "presumed competence" derived from isolated medical board
examinations without longitudinal residency guardrails, (2) distinguishing bounded, lesion-validated
(or feature-validated) CNN architectures from unconstrained VLMs, (3) incorporating Verification
Friction and real-world User Verification Capacity into risk assessment, (4) requiring rigorous
premarket VLM validation—including injected-error rejection and large-scale real-world testing
rather than artificial laboratory MRMC studies, and (5) using a Structured Clinical Taxonomy for
both premarket verification envelopes and continuous postmarket telemetry across millions of
exams, CDRH can establish a clear, scientifically grounded framework for generative AI in
medicine.

The DeepHealth team appreciates the Agency’s collaborative approach and welcomes the
opportunity to share additional empirical data from our clinical deployments to assist CDRH in
developing forthcoming guidance.

Respectfully submitted,

The DeepHealth Team