Khaled Taj

Docket No. FDA-2026-N-7874. Submitted as an individual, in a personal capacity, not on behalf of any employer. Responds to Section IV Questions 1–6 and Section V Question 9. The full comment is attached as a PDF.

Summary recommendation: before the Sec. V competency assessment applies, FDA should require a documented determination, for each safety-critical output element of a generative AI-enabled medical device, of whether its ground truth is data-invariant or interpretive. Data-invariant elements (correct value fixed by a single authoritative source, such as approved labeling, a validated computation, or a fixed decision rule) should be shown safe by closure verification against that source. Interpretive elements (where competent specialists could reasonably disagree) should keep benchmarking followed by clinical confirmation, with the benchmark’s independence from the model documented.

The attached comment also addresses: reasonably foreseeable misuse and automation bias from fluent generative output (Q1); modality and negation as separate control categories (Q2); patient-facing comprehension risk (Q3); when clinician review can be credited as a risk control (Q4); safety properties across multi-turn conversations (Q5); the two classes of over-escalation and their link to under-escalation (Q6); and why benchmarking and closure verification support different claims, including evidence that survives a change of foundation model (Q9).

Docket FDA-2026-N-7874 — Generative AI-Enabled Medical Devices
Khaled Taj, in a personal capacity; not the views of any employer. Responds to Sec. IV Q1–Q6 and Sec. V Q9.

Summary recommendation: require, for each safety-critical output element, a documented determination of
whether its ground truth is data-invariant or interpretive before the Sec. V competency assessment applies —
with closure verification for data-invariant elements and benchmarking followed by clinical confirmation for
interpretive ones.

1. Governing principle: data-invariance
A safety-critical output element is data-invariant where its correct value derives from a single authoritative
source — approved labeling, a validated computation, or a decision rule with fixed inputs and outputs — so
two competent evaluators cannot differ. Dosages and part identifiers qualify. It is interpretive where the
answer requires synthesizing conflicting signals into a judgment on which competent specialists could
disagree. The determination is per output element, not per device; a single turn typically contains both. Below,
the reference is that authoritative source, and the encoded reference is its machine-readable representation
used by the device.

2. Sec. IV, Q1 — Risk axes and generative delivery
The two axes miss reasonably foreseeable misuse (ISO 14971) attributable to generative delivery itself:
over-trust, or automation bias, produced by fluent, personalized output. Fluency gives an inference the
appearance of a verified result, so users rely on estimates as though they had been checked. This cannot be
scored on any single output, because the output is not fixed; it is a property of the delivery mode and should be
assessed across the range of outputs and the intended user population.

3. Sec. IV, Q2 — Directiveness continuum
Modality (“may want to,” “should,” “must”) changes directive force without changing clinical content, and
can move an output from non-directive to action-directing information without any factual error. Negation
scope is a separate failure: a misscoped negation fluently directs the opposite action. Explicit directive force is
carried by an enumerable set of linguistic markers, so a device can constrain it by design rather than estimate it
from samples — a necessary control, not a sufficient one, since plain statements of fact can direct action
without modal marking.

4. Sec. IV, Q3 — Patient-facing delivery
Risk rises materially through probability of harm: ordinary terms carry different meanings across populations,
so correct output is misread more often. Severity shifts where misreading routes the user into a different action
with a different worst case. Mitigation should be calibrated: interpretive and procedural language can remain
flexible, since adapting language to the user is where the access benefit arises, while data-invariant elements
remain fixed.

5. Sec. IV, Q4 — Generalist vs. specialist HCPs
Partial domain competence can be less protective than absent competence: it gives enough confidence to
accept an output without the depth to detect error. Specialist expertise concentrates in rare presentations,
where a model trained on common patterns is fluent, conventional, and wrong. Performance assessment
should weight rare presentations, not average performance. Two safeguards follow: validate that users
perceive the device as a subordinate aid, and flag output within a specialty outside the clinician’s scope.
Separately, clinician review should be credited as a risk control only to the extent the reviewer’s judgment is
independent of the output: a reviewer who reads the answer before forming a view is subject to anchoring and
automation bias, and shares the output’s errors.

6. Sec. IV, Q5 — Multi-turn trajectories
Risk emerges across the conversation even where no single turn becomes action-directing: a device can pass
every per-turn check while the exchange drifts or incrementally weakens an earlier warning. Data-invariant
elements must remain correct across the conversation, with restatements checked against the reference rather
than against the preceding turn. Intended use should be characterized by the safety properties the device
maintains across a conversation, not by enumerating emergent behavior.
7. Sec. IV, Q6 — Under- and over-escalation
Under- and over-escalation are hard to weigh against each other partly because over-escalation is treated as
one error class. It is two. Scope rejection carries little clinical cost; clinical referral consumes care capacity and
desensitizes users, and the two should not be weighed on the same scale. Nor are under- and over-escalation
independent: excessive referral degrades responsiveness to escalation, so beyond a threshold over-escalation
causes under-escalation. Where intended use defines scope as a finite list of coded concepts (e.g., in
SNOMED CT), checking whether an identified concept is on that list is data-invariant. Identifying which
concept a free-text query expresses is not: that mapping is interpretive, and like the escalation boundary it
requires validation against prespecified acceptance criteria, set from the consequences of each error class
before validation data are seen.

8. Sec. V, Q9 — Benchmarking structure
Benchmarking suits interpretive output and should not be weakened; it is misapplied only when used without a
prior data-invariance determination. For data-invariant elements, a different instrument applies: closure
verification, which specifies the complete set of permitted outputs and verifies every output against the
encoded reference. The two instruments support different claims. A benchmark yields a performance estimate
across the sampled cases; for data-invariant elements that assurance is statistical and limited to the cases
sampled, at any sample size. Closure verification covers the whole output space by design, and checking each
output against the encoded reference verifies that individual output before it is displayed — so benchmarking
there is redundant, and strictly weaker. Because the reference is external to the model, this evidence also
remains valid when the foundation model changes, whereas benchmark evidence must be re-established or its
transfer justified, for example within a Predetermined Change Control Plan.
Each instrument carries a condition. Agreement with a benchmark establishes correctness only for errors the
benchmark does not share with the model: overlap between benchmark items and the model’s training data,
scoring by a language model, and items curated or generated by a model can each produce agreement without
correctness, so the independence of the benchmark from the model under test should be documented. Closure
verification shows conformance to the encoded reference, not to the reference itself; the encoding requires
independent verification, and that is the one step closure verification cannot perform.
Two standards already impose this ordering: ISO 14971 requires inherently safe design before protective
measures, because a control that eliminates a hazard by design outranks one that manages residual risk; HL7
SPL carries FDA-regulated labeling in a published, machine-checkable structure verified by automated
conformance checks, separating structural verification from review of clinical content — the same division
this comment proposes between data-invariant and interpretive elements.

Recommendation. FDA should require the following:
1. Determination first. For each safety-critical output element, the manufacturer documents whether its
ground truth is data-invariant or interpretive, before the Sec. V competency assessment applies.
2. Data-invariant elements: closure verification. Require all four of the following; none is sufficient
alone: (a) a published, machine-checkable specification of the permitted outputs (for example, a schema for
structured data, a formal grammar for text, or permitted values and units for numeric output); (b) proof that
the specification is complete (it covers every case in the defined output space) and internally consistent (no
two rules conflict); (c) a conformance test suite; and (d) independent verification that the encoded reference
matches the reference. Each closes a different gap: the specification defines which outputs are permitted;
the proof shows it covers every case without conflict; the conformance tests show the device’s outputs meet
the specification; and verification of the encoding shows the specification reflects the reference, not an
error made in transcribing it. Where output is rendered as speech or images, the specification applies to the
content before rendering, and the rendering step requires its own verification. Benchmarking of these
elements is then redundant and cannot substitute for closure verification.
3. Interpretive elements: the Sec. V competency assessment. Require benchmarking followed by clinical
confirmation, with the independence of the benchmark from the model under test documented.
4. Burden in both directions. A manufacturer that classifies an element as data-invariant provides the
evidence in point 2; one that classifies it as interpretive shows plausible alternative ground truths or
published evidence of expert disagreement.