Tanisha Tejada Thoreau

Executive Overview

The Center for Devices and Radiological Health (CDRH) is to be commended for its forward-thinking discussion paper regarding Generative AI (GenAI)-enabled medical devices. The Agency rightly notes that traditional static software validation paradigms—predicated on closed input/output bounds—fail to account for the stochastic, multi-task, and emergent properties of foundation models and generative clinical architectures.

In my companion work, The Algorithmic Audit Doctrine (AAD), I address this exact regulatory dilemma within GxP and clinical workflows. My thesis posits that because generative algorithms operate as quasi-autonomous analytical agents, governance must shift from static documentation and paper compliance to real-time technical enforcement, runtime boundary defense, continuous proficiency benchmarking, and immutable data lineage.

While CDRH’s proposals move regulatory science in the right direction, the current discussion paper contains critical technical omissions regarding runtime threat defense, data integrity (ALCOA+), and third-party foundation model risk. This comment outlines specific points of agreement, key areas of divergence, actionable mechanisms to strengthen the framework, and answers to specific docket discussion questions.

See attached file(s) for remainder.

MEMORANDUM FOR PUBLIC DOCKET SUBMISSION

Docket No.: FDA-2026-N-7874

Executive Overview & Alignment

The Center for Devices and Radiological Health (CDRH) is to be commended for its forward-thinking
discussion paper regarding Generative AI (GenAI)-enabled medical devices. The Agency rightly notes that
traditional static software validation paradigms—predicated on closed input/output bounds—fail to account for
the stochastic, multi-task, and emergent properties of foundation models and generative clinical architectures.

In my companion work, The Algorithmic Audit Doctrine (AAD), I address this exact regulatory dilemma within
GxP and clinical workflows. My thesis posits that because generative algorithms operate as quasi-autonomous
analytical agents, governance must shift from static documentation and paper compliance to real-time technical
enforcement, runtime boundary defense, continuous proficiency benchmarking, and immutable data lineage.

While CDRH’s proposals move regulatory science in the right direction, the current discussion paper contains
critical technical omissions regarding runtime threat defense, data integrity (ALCOA+), and third-party
foundation model risk. This comment outlines specific points of agreement, key areas of divergence, actionable
mechanisms to strengthen the framework, and answers to specific docket discussion questions.

1. Areas of Substantive Agreement
• Adoption of Competency-Based Assessment (Section V): CDRH’s proposal to model premarket
evaluation on clinical credentialing, medical licensing, and structured competency rather than exhaustive
combinatorial testing is scientifically sound and operationally necessary. In AAD Pillar 2 (Algorithmic
Competency Assurance), I argue an identical parallel: algorithms in regulated settings must be subjected
to ongoing proficiency testing—analogous to ISO/IEC 17025 laboratory quality standards—rather than
relying solely on initial static validation.

• The Two-Axis Risk Matrix (Section IV & Figure 1): Mapping "Device Activity" against "Consequences
of Erroneous Output" accurately captures the operational risk gradient. I strongly endorse CDRH’s
recognition that "informational" outputs can become de facto "action-directing" depending on linguistic
formulation, context, and patient vulnerability.

• Shift toward Postmarket Reliance (Section VI): Recognizing that premarket testing cannot predict every
conversational trajectory, CDRH appropriately emphasizes continuous postmarket surveillance and
Predetermined Change Control Plans (PCCPs).

2. Critical Gaps & Areas of Disagreement
A. Disagreement: Permitting "Greater Premarket Uncertainty" Without Hard Runtime Controls
CDRH poses whether the Agency should "accept greater premarket uncertainty regarding a GenAI-enabled
device's benefit-risk profile through greater reliance on postmarket monitoring" (Section VI, Question 18).

• The Critique: In clinical practice and GxP execution, accepting increased premarket uncertainty without
mandatory, automated runtime circuit breakers is hazardous. Postmarket monitoring that relies on
retrospective complaint reviews or periodic sample audits will fail to prevent acute clinical harm or
systemic unblinding before regulatory intervention can occur.

• The AAD Standard: Reduced premarket evidence must be strictly contingent upon the presence of active
runtime boundary controls and automated kill-switches (AAD Pillar 3: Algorithmic Materiality). If an
algorithm displays confidence score collapse or semantic drift, it must be programmatically locked at the
API layer immediately—not weeks later during periodic review.
B. The Omission of Indirect Prompt Injection & Adversarial Ingestion Defense
CDRH mentions prompt injection only in passing within Appendix A (S.2, R.1, A.1). This vastly
underestimates the primary attack vector facing GenAI in clinical ecosystems.

• The Vulnerability: When GenAI devices read unstructured electronic health records (EHRs), external
laboratory PDFs, or patient-submitted portal notes, they are acutely vulnerable to indirect prompt
injection (e.g., adversarial white-text, zero-width instructions, or hidden semantic overrides). A thirdparty document can force an LLM to override clinical contraindications or fabricate negative diagnostic
findings without clinician awareness.

• Regulatory Inadequacy: General cybersecurity guidance does not address semantic payload
manipulation. Pre-ingestion file sanitization must be elevated to a primary design control requirement.

C. Inadequate Data Integrity & Evidentiary Traceability (The Part 11 / A-CoC Gap)
The discussion paper’s postmarket monitoring section lacks technical specificity regarding data integrity
(ALCOA+) and legal defensibility. Under 21 C.F.R. Part 11 and 21 C.F.R. § 820.70, electronic records must
have tamper-evident, independent audit trails. If a generative device recommends a contra-indicated therapeutic
or an agentic device executes an autonomous order, how does an FDA BIMO investigator or court reconstruct
the transaction? Relying on ephemeral cloud vendor logs is insufficient.

3. Actionable Proposals to Strengthen the FDA Framework
Drawing from The Algorithmic Audit Doctrine (AAD), I urge CDRH to integrate four technical requirements
into its eventual draft guidance:

Proposal 1: Mandate Algorithmic Chain of Custody (A-CoC) for Regulated Outputs
(Strengthening Section VI & Addressing Part 11 Compliance)

FDA should require every GenAI-assisted clinical decision or diagnostic output to programmatically bundle and
immutably commit a 6-Point A-CoC JSON Record directly to Write-Once-Read-Many (WORM) storage:

1. Model Identification: Commercial name and unique deployment/tenant ID.

2. Exact Version & Weight Hash: Digital cryptographic fingerprint (e.g., SHA-256) of model weights.

3. System & Context Prompt: Verbatim record of system instructions and user inputs.

4. RAG Data Lineage: Explicit uniform resource identifiers (URIs) of all EHR records, vector database
nodes, or clinical guidelines retrieved.

5. Algorithmic Confidence Metric: Mathematical certainty or token log-probability.

6. Human-in-the-Loop (HITL) Part 11 Signature: Authenticated electronic sign-off from the reviewing
clinician/expert confirming substantive verification.

Proposal 2: Codify Automated Risk-Tiered Escalation & Fail-Closed Cut-Offs
(Strengthening Section IV & Section VI.C)

Incorporate an operational 3-Tier Materiality Matrix that moves beyond passive monitoring to active
programmatic containment:
 • Tier 1 (Low Materiality): Minor stylistic/linguistic variations; routine WORM audit logging.

• Tier 2 (Moderate Materiality): Workflow routing or metadata filtering anomalies; triggers 24-hour
staged quarantine and mandatory human quality unit review.

• Tier 3 (High Materiality): Diagnostic integration shifts (> 1.5%), critical drug-interaction omissions,
unblinding breaches, or prompt injection detections. Mandatory Action: Automated API token
revocation (kill-switch), immediate operational lockout, formal CAPA root-cause investigation, and
mandatory 24-hour FDA/sponsor reporting under 21 C.F.R. § 803 / § 812.150.

Proposal 3: Address Vendor Foundation Model Evolution via Contractual Staging
(Strengthening Section VII.A & Question 24)

Voluntary Foundation Model Device Master Files (MAFs) are helpful but insufficient because model
developers lack economic incentives to maintain them for niche medical device clients. CDRH should require
device sponsors to contractually enforce a 14-day advance notification and preview sandbox window with their
cloud LLM vendors. During this 14-day window, the sponsor’s internal audit unit runs automated, locked
"golden test sets" (proficiency tests) in a sandboxed staging environment to verify that third-party weight
updates have not degraded clinical reasoning before deployment to live clinical workflows.

4. Direct Feedback on Key Discussion Questions
• On Question 1 (Two-Axis Risk Framework):

Add "Output Reversibility" and "Traceability" to the matrix. An output that triggers an irreversible
surgical or pharmacological event must carry higher evidentiary weight than one with downstream
clinical verification buffers.

• On Question 7 & 10 (Benchmarking & Evaluation Assets):

Public benchmarks face severe data saturation and leakage. CDRH should require sequestered, sponsorcurated ground-truth test sets evaluated by structurally independent clinician panels. Benchmarks should
incorporate simulated clinical friction: noisy measurements, contradictory patient histories, and edge
cases.

• On Question 20 (Machine-Based Supervisory Agents):

Automated LLM "evaluator agents" can assist in high-throughput postmarket monitoring, but an
algorithm cannot serve as the accountable decision-maker under 21 C.F.R. § 211.22 or 21 C.F.R. §
820.20. Any supervisory agent must itself undergo competency qualification, operate under strict
deterministic rules, and remain subordinate to credentialed human oversight.

• On Question 26 (Agentic AI Systems):

Agentic devices that execute autonomous tool sequences or issue device controls present catastrophic
risk multiplication. CDRH must enforce mandatory, non-bypassable HITL checkpoints prior to any
irreversible action (e.g., dispensing medication, issuing orders, changing infusion rates) and require
isolated vector index partitioning to prevent cross-tenant data bleed.

Conclusion
CDRH’s discussion paper establishes a thoughtful foundation for generative AI oversight. However, clinical
safety and statutory compliance cannot depend on trust or after-the-fact monitoring alone. By adopting the
principles outlined in The Algorithmic Audit Doctrine—continuous ingestion filtering, blind competency
benchmarking, quantitative materiality kill-switches, and immutable chain of custody—the FDA can provide an
agile, least burdensome regulatory pathway that protects public health, preserves data integrity, and fosters safe
clinical innovation.

Respectfully submitted,

JD Candidate