Javid Rahaman

Please see the attached comment letter responding to the discussion paper on Considerations for the Regulation of Generative AI-Enabled Medical Devices (Docket FDA-2026-N-7874).

I submit these comments as an individual practitioner and researcher in applied and agentic AI (PhD candidate in Reinforcement Learning and AI; research on dynamic treatment regimes and clinical AI; 21 years in enterprise architecture and data engineering). The views are my own.

Summary of recommendations:
1. Add reversibility, output traceability, and sequential dependence as explicit risk modifiers within the two-axis framework (Q1).
2. Assess multi-turn and agentic devices at the trajectory level, defining intended use as an envelope of permitted states and actions (Q5, Q26).
3. Bind evaluation, change control, and re-benchmarking to a versioned AI configuration bill of materials (model version, prompts, retrieval corpus, guardrails, orchestration, tools) (Q7, Q22-24).
4. Pair any reliance on postmarket evidence with prespecified sequential drift detection, sampled independent clinician adjudication, and demonstrated rollback (Q18-20).
5. Keep synthetic-data evidence separate and model-family independent, and gate benchmarks on demonstrated construct validity (Q10, Q12, Q13).
6. For agentic devices, require tiered action authorization, least-privilege tool access, prompt-injection testing, and complete plan/tool-call audit logs (Q26).

Detailed responses to Questions 1, 5-7, 10, 12, 13, 16, 18-20, and 22-26 are in the attachment.

October 12, 2026
Dockets Management Staff (HFA-305)
Food and Drug Administration
5630 Fishers Lane, Rm. 1061
Rockville, MD 20852
Re: Docket No. FDA-2026-N-7874 — Considerations for the Regulation of Generative
AI-Enabled Medical Devices: Discussion Paper and Request for Feedback
To the Center for Devices and Radiological Health, Digital Health Center of Excellence:
Thank you for the opportunity to comment on this discussion paper. I submit these comments as an
individual practitioner and researcher in applied and agentic AI. The views below are my own and do
not represent any employer or institution.

Submitter background
• Vice President & Chief Architect, Cloud & Generative AI, Rapidflow Inc.
• Founder & Head of Applied Intelligence, AAI Applied Agentic Intelligence Research Labs®,
focused on agentic AI systems.
• PhD candidate in Reinforcement Learning and AI, Golden Gate University; doctoral research on
dynamic treatment regimes and clinical AI decision systems, with a time-series focus.
• 21 years at Oracle in senior enterprise-architecture and data roles (VP level); TOGAF 9.1 certified;
B.Eng. in Computer Science.
• Author, Modern Oracle Enterprise Architecture (BPB Publications, 2022); named inventor, U.S.
Patent Application Publication No. US 2021/0382868 A1.

My comments concentrate on the questions where this background is most relevant: sequential
(multi-turn and agentic) risk, benchmark validity and synthetic data, postmarket drift monitoring, and
configuration and change control for systems built on third-party foundation models. I support CDRH's
total product life cycle orientation and the competency-based approach, and offer the following
recommendations to make them operational and auditable.

Summary of recommendations
• Add reversibility, output traceability, and sequential dependence (whether an output conditions
later decisions) as explicit risk modifiers alongside the two axes.
• Assess conversational and agentic devices at the trajectory level, not the turn level, and define
intended use as an envelope of permitted states and actions.
• Require a versioned AI configuration bill of materials (AI-BOM) as the unit of authorization,
change control, and re-benchmarking.
• Pair postmarket reliance with prespecified sequential drift detection, sampled independent
adjudication, and demonstrated rollback.
• Keep synthetic-data evidence separate and model-family independent, and gate benchmarks on
demonstrated construct validity.
• For agentic devices, use tiered action authorization, least-privilege tool access, and complete
plan/tool-call audit logs.

Section IV — Assessment of risk
Question 1 — Additional risk dimensions.
The two-axis framework is a sound heuristic. I recommend three modifiers that shift a function along
the consequences axis rather than adding new axes, which preserves the framework's simplicity:
• Reversibility and time-to-detection. An erroneous output whose effect can be caught and reversed
before harm (e.g., a draft order requiring signature) warrants less evidence than one that triggers an
irreversible or time-critical action.
• Traceability. Outputs grounded in retrievable, cited primary sources that the user can check carry
lower risk than unsupported free text. Traceability should be credited only when citation accuracy is
itself benchmarked.
• Sequential dependence. In the dynamic-treatment-regime literature, a decision's risk depends on
how it conditions later decisions. A titration recommendation that becomes the starting point for
subsequent recommendations compounds error; a one-off informational output does not. Functions
that feed their own later inputs should be rated higher.

Question 5 — Multi-turn trajectories.
A multi-turn device is best analyzed as a sequential decision process. I recommend that sponsors (a)
define intended use as the set of permitted states and output classes the device may reach, (b) evaluate
the device against a library of simulated user policies (cooperative, anxious, minimizing, adversarial,
low health literacy) run over many turns, and (c) report trajectory-level metrics such as the maximum
directiveness reached, the turn at which scope was exceeded, and the rate of unrecovered drift. A
per-turn scope classifier that is itself validated can serve as a runtime guardrail and as a monitoring
signal postmarket.
Question 6 — Under- versus over-escalation.
These errors should not be collapsed into a single accuracy figure. Sponsors should prespecify a
context-specific loss matrix elicited from clinicians, report sensitivity and specificity for escalation at
the chosen operating point with confidence intervals, and justify the operating point against that matrix.
Over-escalation should also be monitored longitudinally, since its harms (loss of trust, reduced
appropriate care-seeking) emerge over time.

Section V — Competency-based premarket evaluation
Question 7 — Overall approach.
Benchmarking followed by clinical confirmation is appropriate. Its effectiveness depends on evaluating
the exact deployed configuration. I recommend that the unit under evaluation be defined by an
AI-BOM: foundation model identifier and immutable version, system prompts, retrieval corpus
snapshot and indexing parameters, guardrail and safety-classifier versions, orchestration logic, tool
definitions, and decoding parameters. Any benchmark result should be bound to an AI-BOM hash.
Questions 10, 12 and 13 — Benchmark validity and synthetic data.
• Contamination control. Use sequestered holdout sets, rotated on a schedule, plus temporal holdouts
drawn from data created after the foundation model's training cutoff. Canary items can detect
leakage.
• Construct validity. A benchmark should gate evaluation only if its scores have been shown to track
clinical-confirmation outcomes for the intended use. Sponsor-developed benchmarks are valuable
for device-specific behavior but should be complemented by an independent sequestered set.
• Synthetic data. Test inputs should not be generated by a model of the same family as the device
under test, because shared blind spots will not be exposed. Sponsors should quantify the gap between
synthetic and real input distributions, report synthetic-data results separately, and pool estimates only
when distributional equivalence on clinically relevant features has been demonstrated. Synthetic data
is best suited to stress-testing rare safety-critical presentations and adversarial behavior; it is least
suited to establishing subgroup performance for populations that are underrepresented in the data
used to build the generator.
• Statistical rigor. Because outputs are stochastic, report variance across repeated runs and
paraphrases, and power analyses on the safety-critical failure rate rather than on average quality
scores.

Question 16 — Independent third parties.
Maintaining sequestered datasets and supplying independent adjudication are the highest-value
third-party roles. To avoid limiting competition, recognized test assets and methods should be openly
specified even when individual test items remain sequestered.

Section VI — Postmarket monitoring and change control
Questions 18 and 19 — Postmarket reliance and monitoring methods.
Accepting greater premarket uncertainty can be appropriate for informational and HCP-supervised
functions when the monitoring program includes: prespecified performance thresholds with defined
actions; continuous drift detection on input distributions, output characteristics, and safety-classifier
rates using sequential methods (e.g., CUSUM or sequential probability ratio tests with error-rate control
for repeated looks); periodic, stratified sampling of real interactions for independent clinician
adjudication; and a demonstrated ability to roll back to the last authorized configuration. It should not
substitute for premarket evidence for autonomous action-taking functions in the highest-consequence
tier. Re-benchmarking should be triggered by any AI-BOM change and by drift alarms, in addition to a
fixed cadence.
Question 20 — Machine-based supervisory agents.
Supervisory agents can scale triage of interactions for human review. They should be from a different
model family than the device, validated against human adjudication on a held-out sample with reported
agreement, periodically recalibrated, and never the sole monitor for high-risk functions. Their own
versions belong in the AI-BOM.
Questions 22–24 — Modifications and third-party foundation models.
• Change classes. Classify changes by AI-BOM component and expected impact: (1) non-behavioral
(infrastructure, logging) managed in the QMS; (2) behavioral changes within a validated envelope
(retrieval corpus refresh, prompt edits, like-for-like model version upgrades) eligible for a PCCP
with full re-benchmarking against the premarket battery; (3) changes to intended use, autonomy
level, or tool access requiring FDA review.
• Unspecifiable future changes. A PCCP can prespecify the verification protocol and acceptance
criteria rather than the change itself, provided the re-benchmark battery and thresholds are fixed and
results are documented before deployment.
• Third-party model changes. Manufacturers should pin immutable model versions, obtain
contractual advance notice and deprecation timelines, run shadow or canary evaluation of a new
version before cut-over, and monitor for silent behavioral changes by tracking responses to a fixed
probe set at a regular cadence.

Section VII — Other topics
Question 25 — Foundation Model Device Master Files.
Voluntary MAFs are practical if a structured template is provided. The most useful contents for review
are version and deprecation policy, update-notification commitments, documented safety behaviors and
known failure modes in healthcare contexts, healthcare benchmark results with subgroup breakdowns,
and audit-log availability. Participation could be encouraged by allowing referenced MAFs to reduce
the documentation burden in sponsor submissions.
Question 26 — Agentic GenAI-enabled devices.
• Tiered action authorization. Classify each tool or action by reversibility and consequence, and
require human confirmation before any irreversible or high-consequence action; the permitted
autonomy level should be part of the authorized intended use.
• Least privilege. Grant tools the minimum scope needed, and treat expansion of tool access as a
change requiring review.
• Prompt-injection resistance. Content returned by retrieval and tools must be treated as data, never
as instructions; this should be tested adversarially as part of benchmark element A.1.
• Auditability. Log the full plan, each tool call and result, and each human checkpoint, so that
postmarket review can reconstruct any trajectory.
• Acceptance criteria. Set criteria on end-to-end task trajectories, including the rate of unsafe
intermediate actions and recovery from erroneous tool outputs, not only on final outputs.

Conclusion
CDRH's direction is well founded. Binding evaluation to a precisely versioned configuration, assessing
risk over trajectories, and pairing any reduction in premarket evidence with rigorous, prespecified
postmarket monitoring would make the framework both least burdensome and auditable. I would
welcome the opportunity to support further work on these questions.
Respectfully submitted,

Javid Rahaman
Founder & Head of Applied Intelligence, AAI Applied Agentic Intelligence Research Labs®
Irving, Texas