Counsel Health (Cian Hughes)

See attached document

Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Comments on CDRH Discussion Paper:
Considerations for the Regulation of
Generative AI-Enabled Medical Devices
​
Re. Docket No. FDA-2026-N-7874

Dear Dr. Tarver and Dr. Abramson,
Counsel Health appreciates the opportunity to comment on CDRH's discussion paper,
Considerations for the Regulation of Generative AI-Enabled Medical Devices (Docket No.
FDA-2026-N-7874). Counsel provides AI-enabled, physician-delivered care, combining
generative AI with board-certified physicians in pursuit of our mission to multiply the world's
clinical capacity. How generative AI is evaluated, monitored and allowed to change over time
bears directly on that mission, and we welcome CDRH's decision to consult openly before issuing
guidance. Our detailed responses follow the numbering of the discussion paper's questions.

Summary of recommendations
We support a rigorous, postmarket-focused approach to the safety and evaluation of generative
AI-enabled devices. Our central recommendation is that, for low-acuity, reversible,
clinician-supervised functions, competency benchmarking plus an auditable monitoring plan
should be sufficient premarket evidence, with clinical confirmation generated in supervised
operation after authorization (Q18). Six recommendations build the framework around that:

1.​ Authorize multi-indication, competency-based devices, with a competency envelope as
the unit of authorization and per-function performance floors inside it (Q5, Q8, Q23).
2.​ Require premarket testing in silico rather than on patients: digital twins built from
physician practice records and from fully synthetic cases, covering rare presentations and
underrepresented groups, as a premarket gate (Q11, Q13).
3.​ Accept prospectively designed real-world evidence from supervised operation as the
main clinical-confirmation evidence once a device is live (Q7, Q11, Q14).
4.​ Keep the simulation-based regression gate for every release and every
foundation-model change, so that the premarket test and the change-control test are
one instrument (Q22 to Q24).
5.​ Permit an approach where devices move through deployment phases with increasing
levels of autonomy (e.g. real-time supervision; near-real-time asynchronous review;
autonomy with sub-sampled asynchronous review), with gates set in advance and
functions within one device able to sit in different phases at once (Q18, Q26).
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

6.​ Treat authorization as an ongoing obligation: prespecified performance floors, mandatory
reporting, required reversion when performance degrades, and change control defined
by tests rather than by enumerated changes (Q18 to Q20, Q23).

In support of these, we propose four testable modifiers within the two-axis risk framework (Q1); a
directiveness ladder with clear criteria for enforcement discretion in supervised, patient-facing
care (Q2); a hard recall floor and a monitored over-escalation ceiling for escalation functions
(Q6); and (as an idealized end-state) a blinded, independently held clinical benchmark to which
sponsors submit their systems rather than their scores (Q16).​

We share CDRH’s optimism for the potential of Generative AI enabled medical devices to
improve health outcomes and the delivery of healthcare to all Americans. We note that the
oversight CDRH is describing for agentic devices is the oversight the profession already applies
to human trainees: graduated autonomy, supervision proportionate to demonstrated
competence, and revocation when performance falls. A framework that follows that model will
be recognizable to clinicians and workable for sponsors. We would welcome the opportunity to
discuss these proposals with the Center.

Respectfully,

Dr. Cian Hughes
Chief Scientific Officer
Counsel Inc.

​
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Detailed Responses
Section IV — Risk

Q1. Two-axis risk framework
Does the two-axis framework (device activity × consequence of relying on an incorrect output)
capture the right dimensions? Should reversibility, downstream safeguards, time pressure or
traceability be added?

The two axes are the right primitives and we support them. We propose four modifiers that act
within the framework rather than as new axes, because each is testable and each changes where
a function sits on the consequence axis. Each modifier should be explicitly defined, measurable,
and supported by evidence demonstrating its effectiveness in reducing clinical risk.

Reversibility: whether the output can cause a real-world action before a human can intervene. A
device that produces a plan which a licensed clinician must approve is on a different
consequence tier from one that can transmit a prescription, even if the text of the plan is
identical. Downstream safeguards: whether a qualified physician is reachable inside the same
interaction, and within what time bound. We suggest this be expressed as a time rather than a
yes/no, because a safeguard reachable in minutes and one reachable in days are different
mitigations. This matters for a deployment model we expect to become common: the device
responds to the patient directly and a licensed clinician reviews the exchange asynchronously
within a bounded window, able to correct the response and to hold any downstream action until
review completes. Experience in deployed asynchronous telehealth indicates that clinician
review within a few minutes is achievable at scale, so a short, enforceable window is a practical
requirement rather than a theoretical one. Time pressure: whether the deployment setting
admits delay. Functions that route time-critical presentations out of the device entirely should
be assessed on the residual population. Traceability: whether clinicians can independently verify
the information underlying an output, including relevant patient data, retrieved clinical evidence,
and source provenance. Traceability should support meaningful clinical review rather than
relying on generated explanations of model reasoning.

Taken together, these modifiers describe a recognizable configuration: a device that proposes
but cannot act, a qualified clinician reachable inside the same interaction within a measured
window, time-critical presentations routed out to emergency services, and sources and
reasoning visible to the reviewing clinician. We would welcome a framework that lets a sponsor
claim credit for each of these controls explicitly, with the evidence to show that each is enforced
in code rather than in labeling.

Q2. Directiveness spectrum
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

How to account for informational functions that vary from non-directive to action-directing?
What output characteristics modify risk? What would give manufacturers clarity?

Directiveness is the right lens, and we agree it is a continuum. We propose that CDRH publish a
directiveness ladder with testable rungs, so that a sponsor can show by evaluation which rung a
function is designed to reach and that it does not exceed it. One workable ladder is: (1) general
health education; (2) population-level guidance ("people with these symptoms are usually
advised to…"); (3) individualized assessment that names a likely condition; (4) individualized
recommendation of a treatment class; (5) a specific treatment, dose and duration; (6) an action
taken on the patient's behalf. Each rung can be detected by a classifier or a judge, which means
conformance can be measured on sampled production conversations rather than asserted.

We suggest that manufacturers may define permissible output boundaries for each function
and demonstrate adherence through validated automated evaluations, including LLM-based
judges calibrated against physician-adjudicated cases. Personalization is an important modifier:
incorporating a patient's history, symptoms, and results can make an output more actionable,
even without explicitly directive wording. In our patient-facing workflow, for example, AI may
explain that urinary tract infections are commonly treated with antibiotics but does not
independently recommend a specific medication or dose; those decisions remain with licensed
physicians. Such boundaries should be demonstrated through system behavior rather than
labeling or disclaimers alone.

To provide manufacturers with greater regulatory predictability, we encourage CDRH to
explicitly define which levels of the proposed directiveness ladder qualify for enforcement
discretion and under what conditions. In particular, we ask CDRH to distinguish between
patient-facing functions operating independently and those integrated into
physician-supervised care, where a qualified clinician is available within the same interaction and
within a bounded timeframe. The practical effect is that a patient minutes away from a clinician's
review can receive less information from a supervised service than from an unsupervised AI tool.
We would very much like to see clear, testable criteria that would permit level 3 (individualized
clinical assessment) under defined safeguards, including reliable emergency recognition and
escalation, restrictions on independent treatment decisions, and physician review within a
specified timeframe with the ability to correct outputs before downstream clinical actions occur.
We believe that such criteria would improve both patient experience and safety by enabling
timely access to personalized clinical information within a supervised care environment, rather
than leaving patients to seek similar information from general-purpose AI assistants without
integrated physician oversight.

Q3. Patient-facing risk
Do patient-facing functions present different or higher risk? What characteristics or safeguards
mitigate risk when the user lacks domain knowledge, without underestimating patients?
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Patient-facing functions carry different risks from HCP-facing functions, but not uniformly
higher ones. A patient may be less able to judge the clinical basis of an output, but knows their
own symptoms, history and circumstances better than any clinician. Patient-facing GenAI can
also widen access to clinical information and improve engagement in care. A framework that
treats every patient-facing function as high risk would discourage the safeguarded versions of
these tools without reducing demand for them. Informational, non-directive functions that help
patients understand their own health, and point them to a clinician where appropriate, sit at the
low end of both axes and should remain there. Clinician supervision should not be a precondition
for that tier. The question this answer addresses is what should be required as a patient-facing
function moves up the directiveness ladder (Q2).

For those functions, the safeguards that work are structural rather than linguistic:

●​ No unsupervised clinical action. The device should not be able to complete an order,
prescription, referral or other clinical action without clinician approval. That way the worst
outcome of an incorrect output is an incorrect statement, not an incorrect intervention.
●​ A clinician within reach. Where a function names a likely condition, a licensed clinician
should be reachable where needed.
●​ Clinician review of individualized recommendations, matched to the evidence. Early in
a function’s life, when prospective outcome data is limited, individualized
recommendations should be reviewed by a clinician within a bounded time. This can take
several forms:
○​ Pre-display hold: the patient sees the recommendation only after a clinician has
confirmed it.
○​ Confidence-gated review: the device pauses to consult a clinician before giving
an individualized recommendation whenever its confidence falls below a validated
threshold, much as a trainee checks with the supervising physician before
answering.
○​ Near-real-time review: the patient sees the response immediately, and a clinician
reviews it within a short window and can correct it before any downstream action
completes.

A pre-display hold suits a function whose safety data is still accumulating. As the function
meets its floors in supervised operation (Q18), it can move to a lighter form of review
without any change to the device’s behavior.

●​ Escalation evaluated independently. Emergency recognition should be evaluated as a
separate function with its own floor (Q6). Otherwise a device’s ability to route a patient to
emergency care is hidden inside its overall conversational performance.

The framework should let a sponsor claim credit for the safety-net provided by each of these
controls, provided it can show that the control is built into the device’s design and holds in
sampled production use.
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Q4. Generalist vs specialist HCPs
Does the generalist/specialist distinction change the risk of HCP-facing functions? When might
risk rise when a non-specialist receives specialist-area information?

We believe that GenAI-enabled CDS provides an opportunity to up-level generalist physicians
enabling them to independently manage moderately complex chronic conditions and crucially
provide holistic care to patients with multiple morbidities. If managed appropriately, this
approach could both increase access to care and improve the patient experience through
reducing the number of different physicians with whom they need to consult.

The variable that matters is not just the recipient's specialty but also whether the recipient can
independently evaluate the basis for the output and critically appraise it. A generalist who is
shown the patient data, the guideline relied on and the reasoning can evaluate a specialist-area
recommendation; a specialist shown only a conclusion cannot. We therefore suggest that risk for
HCP-facing functions be modified by the transparency of the output as well as by the credentials
of the user, and that the device's intended user be stated as a competency ("a clinician who
manages this condition in routine practice") rather than a specialty title. Where an output falls
outside the intended user's competency, the device should say so and route to the appropriate
escalation rather than present the recommendation as actionable.

One route to postmarket surveillance of a solution designed to up-level generalists may be to
have specialist HCP supervision and evaluation of the AI recommendations and the care
delivered by its users.

Q5. Conversational drift
For multi-turn devices that may migrate from non-directive to action-directing over an
exchange, how should risk be assessed across realistic trajectories? How is intended use
characterized when behavior is emergent?

We agree that for multi-turn devices the trajectory, not the individual turn, is the right unit of risk,
and we think this is the most important open question in the paper. A device that is non-directive
at every turn in isolation can still arrive at an action-directing position over a conversation,
because the patient supplies context incrementally and the model narrows accordingly.

We propose that intended use for a conversational device be characterized as the set of
conversational boundaries it enforces in code: the maximum directiveness rung it may reach
(Q2), the conditions and acuity band it may engage with, the triggers that end the conversation
and hand off to a clinician and what it must do when the patient pushes beyond those
boundaries. The envelope need not be uniform. A multi-indication device can declare a different
level of autonomy for different presentations and trajectories: fully autonomous for some
conditions once their floors are met; information-gathering followed by hand-off to a clinician
for others; and little or no AI function with immediate routing to a physician for presentations
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

outside its scope, such as an acute mental-health presentation in a device intended for physical
primary care. The intended use is then the map from trajectory to permitted autonomy, and that
map is what gets tested: a sponsor can run realistic multi-turn simulations, including adversarial
trajectories written by clinicians, and score the proportion of conversations in which the device
stayed inside its declared envelope and handed off where the map says it must.

This is achievable with current tooling: cumulative scope drift can be scored across sampled
conversations alongside per-turn compliance, and risk controls can be registered by trajectory as
well as by module. We would support a requirement that sponsors of conversational devices
report trajectory-level conformance from sampled production conversations, scored by
validated judges, as a standing postmarket measure.

Q6. Under- vs over-escalation
How could manufacturers characterize and weigh under-escalation and over-escalation, given
they may not be commensurable and acceptable trade-offs vary by context?

Under-escalation and over-escalation are not commensurable and should not be collapsed into
a single accuracy figure. We propose they be governed differently: a hard floor on recall for
defined emergency and urgent presentations, which is a gate (a breach triggers reversion to a
more supervised phase and a report to FDA), and a monitored ceiling on over-escalation, which
is a reported quantity with a tolerance set per clinical context.

Over-escalation is a real harm and should be measured as one. It costs patients money and
anxiety, it displaces emergency capacity, and over time it erodes trust in the device's warnings so
that true escalations are ignored. The right measure is downstream rather than classifier-level:
what proportion of escalations resulted in a change of management, and what the
emergency-department utilization of the device's users looks like against a comparable
population.

One design that handles the two errors separately, rather than through a single threshold, is a
two-stage escalation function. The first stage is tuned for sensitivity and accepts a high
false-alarm rate. A second stage asks one or two targeted questions (duration, associated
symptoms, change in severity) and de-escalates flags that clarification resolves, while a defined
set of high-risk presentations (chest pain on exertion, stroke symptoms, suicidal ideation)
bypasses clarification and goes straight to an emergency recommendation. Sampled clinician
review then checks that de-escalations were clinically reasonable. Designs of this kind preserve
sensitivity while reducing the patient-facing cost of over-escalation. We would encourage CDRH
to evaluate escalation functions as a pipeline, with sensitivity measured at the first stage and
precision at the last, and we would support a requirement that sponsors do the same, with the
evaluation set described well enough (size, authorship, enrichment) for the numbers to be
interpretable.

Section V — Premarket evaluation
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Q7. Competency-based approach
Is benchmarking followed by clinical confirmation a useful and appropriate framework?

Yes. Competency benchmarking followed by clinical confirmation is the right framework, and it
reflects how mature clinical AI teams already evaluate. One way of operationalizing the two
stages is to treat them as one evidence stream observed at two points rather than as a
premarket step followed by a postmarket step. Benchmarking is the simulation track, run against
every release. Clinical confirmation is supervised operation, in which every interaction is a paired
observation of what the device proposed and what a licensed clinician decided. This fits a
conversational device deployed with clinicians in the loop; it will not fit every product, and a
device that acts without a supervised phase, or whose outputs are not naturally paired with a
clinician decision, will need a different confirmation design. The second stage should not end at
authorization. A premarket snapshot of a GenAI device ages quickly, because the model, the
prompts and the population all move; continuous confirmation in supervised operation is what
keeps the evidence current. We expand on this in our answers to Q11 and Q18.

Q8. Risk-proportionate evidence
How might the two-axis risk framework determine the level of evidence needed for a premarket
submission?

Evidence should scale with where each function sits on the two axes, and it should do so function
by function inside a single device. We read the paper as open to multi-indication,
competency-based devices, and we strongly support that direction. GenAI agents are inherently
multipurpose; a primary-care agent that could only be authorized one indication at a time would
need dozens of submissions to cover the ordinary case mix, which no sponsor other than the
largest incumbent could afford and which would delay patient access by years. Breadth is also
where the clinical value is: the common presentations are common.

We propose that the unit of authorization be a competency envelope: the set of functions, the
acuity band and the conversational boundaries (Q5) within which the device is permitted to
operate. Inside the envelope, the level of evidence for each function follows the risk framework.
By way of example: a non-directive, low-consequence function (symptom education) might
need benchmarking and a monitoring plan; an action-directing or higher-consequence function
(naming a condition and recommending a treatment class) might need benchmarking,
supervised-operation evidence with prespecified endpoints, and a simulation gate for the rare
presentations that production data will not surface. A mapping of this kind will have exceptions
and nuances, for instance where a low-directiveness function sits in a time-critical setting, and
we would expect the mapping to be agreed per device in a pre-submission rather than fixed in
guidance. Functions in the envelope can sit in different deployment phases at the same time: the
device can be autonomous for one function and fully supervised for another, with the phase of
each function determined by its own evidence against its own floors.
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

This gives CDRH a proportionate review and gives sponsors a path that scales with the device's
capability rather than with the number of ICD codes it touches.

Q9. Benchmark structure
Would the benchmarking structure provide adequate evidence of clinical knowledge, analytic
capability, safety behavior, communication and generalizability? What is missing, redundant or
misplaced? Which external standards could be used?

The proposed benchmarking structure maps closely onto the evaluation suite we run, and we
think it is sound. We suggest four refinements. Reproducibility, which the structure places within
the robustness element, deserves its own quantitative floor: a GenAI device that gives a safe
answer four times in five is not a safe device, and run-to-run variance on safety-critical behaviors
(escalation, refusal, diagnostic conclusion) should be reported as a number, not folded into a
qualitative robustness assessment. Memory and record-retrieval correctness should be
assessed for longitudinal devices: whether the device correctly recalls and uses what the patient
said three turns or three visits ago is a distinct failure mode from clinical knowledge.
Never-events should be tracked as absolute counts with an exact upper confidence bound, not
as rates, so that zero is reported honestly. Judge validation should be a required artifact:
wherever an LLM judge scores a benchmark, the sponsor should report the judge's agreement
with clinician adjudication on a held-out sample, with the judge versioned alongside the device.

On external standards: we support a role for independent, blinded benchmarks held by bodies
with no commercial stake, such as medical boards or licensing organizations, as an end state,
with sponsor-side testing to the same standard in the interim; we set that out in Q16. We do not
recommend naming any existing public benchmark as a standard. Every widely used clinical
benchmark has by now been optimized against by model developers, so performance on it
measures familiarity with the test more than competence at the task; that is the reason we
propose a blinded, independently held benchmark (and, until one exists, sponsor-side testing to
the same standard) rather than an existing one.

Q10. Benchmark validity and sponsor benchmarks
How should a sponsor show that benchmark performance predicts real-world behavior? What
supports construct validity? What role for sponsor-developed benchmarks given independence
concerns?

Construct validity for a benchmark should be established empirically, by showing that
benchmark performance moves with performance in supervised operation. That requires a
production-side measure that is itself clinically grounded. Raw edit rate is not one: in our analysis
of 40,000 AI-draft and physician-final message pairs across four product generations (reported
in a manuscript currently under peer review), physicians edited 47% of drafts but only 32% of all
drafts carried a clinically meaningful change, 9.1% were judged an inappropriate deviation from
standard of care, and 2.7% carried potential for temporary harm or worse had they been sent
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

unchanged. Surface-level edit magnitude only modestly separated clinically meaningful from
cosmetic edits (AUROC 0.71). A three-stage judge that grades edits for clinical meaning,
appropriateness and counterfactual harm, validated against blinded physician adjudication,
converts routine review into a measure a benchmark can be tested against. With that instrument
in place, the construct-validity test is whether a release that scores higher on the benchmark
also shows fewer inappropriate deviations and less potential harm in supervised operation;
across our four product generations, inappropriate deviations fell from 12% to 7% and potential
temporary harm or worse from 4.1% to 2.0%, which is the production trend any gating benchmark
must track. We would support a requirement that sponsors report, for any benchmark used to
gate a decision, how its scores relate to a validated production-side safety measure across
releases.

On contamination and saturation: public benchmarks should be treated as a floor and a sanity
check, not as evidence. Sponsors should supplement them with sequestered, clinician-written
cases that are version-locked, with answer keys that cannot be edited after the fact, and with
validation and test sets that are never used during development.

Sponsor-developed benchmarks are acceptable, and for a device's specific intended use they are
usually better than anything public, provided three conditions are met: provenance is disclosed
(who wrote the cases, from what sources, with what enrichment), adjudication is independent of
the development team, and the benchmark's predictive relationship to production outcomes is
reported. On independence, see our answer to Q16.

Q11. Clinical confirmation approaches
Clinical confirmation might not require a prospective study in every case. How should a sponsor
select and justify an approach proportionate to risk? Where would the lighter approaches be
insufficient? How should the real-world input distribution be considered? Other methods?

We agree that clinical confirmation should not require a prospective study in every case, and we
propose supervised operation as the primary confirmation method for conversational GenAI
devices. In supervised operation the device produces its structured output (assessment, plan,
escalation decision) and a licensed clinician reviews it before anything reaches the patient. Every
interaction is therefore a paired observation: the device's proposal and the clinician's final
decision, captured in the same structured format. From this a sponsor can report edit rates, the
clinical significance of edits graded on a published scale, escalation agreement, and inter-rater
consistency among reviewing clinicians, continuously and on the real input distribution. We have
done this at scale. Our practice has accumulated well over 100,000 paired observations of an AI
proposal and a physician's final decision, and a recent analysis of 40,000 of those pairs, reported
in a manuscript under review, found that most physician edits did not change clinical meaning,
that inappropriate deviations and potential harm were concentrated in a small, identifiable
subset of outputs (recommendation messages far more than summaries), and that both
declined across successive product generations.
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

This is stronger than a prospective study in three ways. It measures the current version of the
device rather than a frozen one. It compares the device to real clinical decision-making on the
same patients rather than to a control condition. And it runs on the actual distribution of
presentations the device will see, which no simulated cohort reproduces. Its weaknesses are
physician variance and rare events, and we address each: cases should be spread across many
clinicians with inter-rater reliability measured and contested cases adjudicated by an
independent panel, and a simulation track built from digital twins of real cases (Q13) should gate
the rare, dangerous presentations that production volume will not surface in time.

A sponsor should select the confirmation approach by the directiveness and consequence of
the function, not by the novelty of the technology. Supervised operation is sufficient where the
clinician's review is genuine and precedes any action; a blinded variant, in which the clinician
records a decision before seeing the device's proposal, removes the automation-bias objection
where that is needed. That variant should not be assumed to be ethical. It is now plausible that
on some tasks a clinician working without AI assistance performs worse than the assisted team
to a degree that would make an unassisted arm unacceptable to patients, so a blinded design
should be preceded by a small simulated pilot, run on digital-twin cases rather than live patients,
to estimate the performance gap before any patient is assigned to it. Supervised operation is
insufficient, and a prospective study appropriate, where the function acts before a human can
review, where harm is irreversible before detection, or where the device is to launch directly into
autonomous operation without a supervised phase. We think the last case should be rare. A
randomized trial should therefore be held in reserve for those cases rather than assumed as the
default; FDA's own direction supports this: the December 2025 final guidance on use of
real-world evidence for medical devices includes expanded indications for use among its
examples of supported uses, and accepts real-world evidence without requiring identifiable
participant-level data. Endpoints, adjudication methods and data-quality controls should be
agreed with FDA in a pre-submission before volume accumulates, so that the data collected is
evidence and not engagement metrics.

Q12. Statistical measurement
How should sponsors achieve statistically meaningful performance measurement? How should
synthetic and real distributions be reconciled, and when can benchmarking and confirmation
evidence be combined into one estimate?

Statistically meaningful measurement starts with reporting synthetic and real-world results
separately and never pooling them into a single headline estimate. The two distributions differ in
prevalence, in presentation and in what the device has been optimized against; a pooled number
is not interpretable. Combining is acceptable only for a stratified estimate in which every stratum
is labeled by provenance and weighted to the real-world distribution, and never for a safety
claim. A related and more useful form of combining is across minor releases of the same device:
because a GenAI device changes often, restricting every estimate to a single version starves it of
volume. We would support pooling supervised-operation evidence across minor releases where
a run chart of the monitored floors shows stability or improvement over the pooled period and
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

where the final version's performance is not out of distribution relative to the versions pooled
with it; a step change in either direction should break the pool.

Enriched evaluation sets (those over-sampling failures or rare presentations) are valuable but
distort prevalence; they should always be run alongside a representative set drawn from
production, and the two reported side by side. Every reported figure should state its
denominator and its sampling definition, including how cases were selected, how many were
excluded and why. Never-events should be reported as absolute counts with an exact upper
confidence bound on the rate, so that "zero in N" is stated with its actual uncertainty rather than
as a rate of zero. This is easy to do, and it aligns with how never-events should be handled
anyway: each one should be individually reportable to FDA under the device's postmarket
surveillance obligations, so the count in a performance report is a cross-check against the
reports already filed rather than a new disclosure.

Q13. Synthetic data
Where is synthetic data well-suited or inadequate? What safeguards stop same-class
generators reproducing the gaps the evaluation should detect, especially for underrepresented
subgroups?

Synthetic data is well suited to the cases production data will not supply in time: rare red flags,
never-events, adversarial inputs and deliberate boundary-testing. It is poorly suited to
establishing performance across subgroups, languages and presentation styles, because a
generator reproduces the distribution it was trained on, including its gaps. Synthetic data
generated by a model of the same class as the device is particularly weak as evidence, since it
shares the device's blind spots.

Our approach is a digital-twin simulation track built from two sources. The first is real physician
practice: cases derived from platform interactions and their clinical records, altered enough to
prevent re-identification but preserving the clinical structure, which anchors the set to the real
case mix and to real clinician decisions. The second is fully synthetic cases, written or generated
to cover what the records lack: rare and dangerous presentations, adversarial trajectories, and
adequate representation of demographic groups, dialects and literacy levels that are
underrepresented in practice data. This produces a regression set that reflects the real
population, can be run against every release candidate and every foundation-model change, and
can be shared with independent reviewers without disclosing patient data. We propose three
safeguards for synthetic data generally. First, generate with a different model family from the
device under evaluation. Models of the same family share training data, failure modes and
stylistic preferences, and a documented self-preference bias means a model is more likely to
rate its own family's outputs favorably and to generate test cases that its own family finds easy;
cases generated by a disjoint family, and judged by a disjoint family (Q20), break that loop.
Second, have clinicians review a sample of generated cases for plausibility and label quality.
Third, always pair synthetic results with representative production sampling (Q12) so that gaps
the generator cannot see are caught by the real distribution.
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Q14. Comparators and acceptance criteria
For open-ended outputs, how should comparators and acceptance criteria be selected? When a
clinician panel is the comparator, is the standard "standard of care" or "median clinician"?
Generalist or specialist? When should human-AI team performance be the basis?

For open-ended outputs the best comparator is the treating clinician's final decision on the same
patient, observed continuously in supervised operation (Q11). It is a real decision made with real
stakes on the real input, which no panel reviewing transcripts reproduces. Where a panel is used,
the applicable standard should be the standard of care, not the performance of a median
clinician in practice. Median performance is a moving and unpublished target that varies by
setting; standard of care is documented, and deviations from it can be graded. Device-clinician
deltas should be graded for clinical significance using published appropriateness and
harm-severity scales (an adapted RAND/UCLA appropriateness method and a seven-tier scale
adapted from the AHRQ harm framework) rather than counting any difference as an error,
because most edits are stylistic: in our data (manuscript under review), 68% of drafts needed no
clinically meaningful change despite a 47% raw edit rate, and physician edits more often
narrowed or de-escalated AI content than added to it. One caution from the same work: in a
300-pair validation sample adjudicated by three blinded physicians, agreement was high on
whether an edit changed clinical meaning but much lower on appropriateness and on
counterfactual harm, so the comparator standard for those downstream judgments needs an
explicit, calibrated reference rather than individual clinician opinion. Larger inter-rater reliability
studies are needed to set the reference ceiling for production floors.

For primary-care functions the generalist is the right standard; the device is replacing or
supporting a generalist's decision, and holding it to a specialist standard on every presentation
would hold it to a standard the counterfactual care does not meet. Specialist review is
appropriate for adjudicating contested cases and for setting the floors on specialist-area
functions within the envelope.

Human-AI team performance should be evaluated whenever the device is deployed with a
human in the loop, because that is the clinical reality; the device alone should be evaluated as
well, because its standalone performance is what the phase-advancement decision rests on.
Reporting both, and the difference between them, also measures automation bias directly: a
team that performs worse than the device alone is a team that is deferring, and that is a signal
the review is not genuine.

Q15. Counterfactual comparators
Could performance be assessed against the care likely to occur absent the device (unaided
judgment, delayed specialist review, no intervention)? How to identify and justify such a
comparator?
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Yes, and we think the counterfactual is under-weighted in device evaluation generally. For a large
share of the people who use a conversational health device, the realistic alternative is not a
timely clinician visit but a search engine, a general-purpose assistant, a delayed appointment, or
no care. A comparator that assumes the patient would otherwise have seen a clinician overstates
the risk of the device and understates its benefit.

We suggest the counterfactual be used as the benefit comparator, alongside rather than instead
of a standard-of-care safety comparator. The safety floors should be set against standard of care
(Q14), so that the device is never held to a lower standard than a clinician on the question of
harm. Benefit, including time to appropriate care, resolution without escalation, and appropriate
escalation that would otherwise not have happened, should be measured against what the
served population would actually have done. Identifying that counterfactual is empirical:
sponsors can report prior care-seeking behavior, time since last contact with a clinician and
stated alternatives from the device's own users. Sponsored or population-level access changes
the counterfactual materially, and the comparator should reflect the population actually served.

Q16. Third-party involvement
Should independent third parties be involved in benchmarking or clinical confirmation? What
qualifications and independence criteria? Where is it impractical? How to stop it limiting
competition?

Independent third parties should be involved, and we propose different roles for benchmarking
and for clinical confirmation.

For benchmarking, the ideal end state is a blinded, universal clinical benchmark held by a body
with no commercial stake, such as a medical board, a licensing body or an organization
analogous to those administering licensing examinations. Sponsors would submit the device for
evaluation rather than reporting their own scores; the case bank would be sequestered and
rotated; and results would be reported on a common scale covering clinical accuracy, safety
behavior, escalation, communication and consistency. This addresses contamination and
optimization to the test at the source, and it gives CDRH a comparable measure across sponsors
that no sponsor-run benchmark can provide. It should complement, not replace,
sponsor-developed benchmarks for a device's specific intended use (Q10).

The mechanism can be concrete. A manufacturer would submit a container image (or an
equivalent packaged artifact) that fully encapsulates the device's core agents and exposes them
through an agreed evaluation API. The benchmark custodian runs the container inside its own
harness against the sequestered case bank and returns scores; neither the cases nor the answer
keys leave the custodian's environment, and nothing about the device beyond the API surface
needs to be disclosed. Containerization also makes the evaluation reproducible: the same image
can be re-run when the benchmark is refreshed, and the image hash ties a reported score to an
exact version of the device. Devices built on hosted third-party models can be accommodated
through a proxy that pins the model version and records it in the run.
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Such a benchmark does not exist yet, and building one will take time; progress should not wait
on it. Until a centralized benchmark is available, sponsors should be able to meet the same
objective through sponsor-side testing designed to the same principles: a case bank
sequestered from the development team and version-locked, with answer keys that cannot be
changed after the fact; test sets never used during development; the system evaluated as a
packaged, versioned artifact through a fixed harness; judges validated against independent
clinician adjudication; and full disclosure to FDA of case provenance, enrichment and results.
Independence can be strengthened short of a central custodian, for example by having an
independent party hold or audit the case bank, or by having a sample of results adjudicated
independently. When a centralized benchmark becomes available, the same packaged system
can be run against it, so a sponsor's interim work carries forward.

Participation should be a choice, not a mandate. Independent benchmarking, whether through a
central custodian or, in the interim, through sponsor-side testing that meets the conditions
above, is offered as a route that may, in defined circumstances, let a sponsor substitute
contamination-free evidence of competence for part of the premarket evidence and so avoid a
prospective clinical study. A sponsor that prefers not to use either route can rely on a
prospective study instead. That keeps the program from becoming a bottleneck or a barrier to
entry, and it means no sponsor's path depends on FDA first standing up new infrastructure.

For clinical confirmation, independence should attach to adjudication rather than to data
collection. The supervised-operation data is generated inside the sponsor's product and cannot
practically be collected by anyone else, but grading of contested cases, inter-rater reliability
measurement and audit of the sampling should be done by clinicians independent of the
development team. We would expect independent adjudication of this kind to be a condition of
any authorization that relies on supervised-operation evidence. Independence criteria should be
published and apply equally to human reviewers and to LLM judges: no involvement in the
device's development, no financial stake in the outcome, and documented validation of the
judge against independent clinician review.

The risk to guard against is capacity. A requirement that every evaluation pass through a small
number of accredited third parties would create a bottleneck that favors incumbents and slows
innovation without improving safety. Published independence criteria that any qualified party
can meet are preferable to a mandated list of vendors.

Q17. Other model architectures
Does the competency-based approach apply to multimodal vision-language models or world
models? What is inapplicable or additional?

The competency-based approach applies per modality: a device that accepts images should be
benchmarked on image-dependent functions separately, with its own floors, and clinical
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

confirmation should record whether the clinician agreed with the device's reading of the image
as a distinct field.

Section VI — Postmarket

Q18. Premarket uncertainty vs postmarket monitoring
Could greater premarket uncertainty be accepted through greater reliance on postmarket
monitoring? Under what conditions, and what must the monitoring program have? Where would
this be inappropriate?

Yes. This is the change we most want to see, and we think it is the only approach that fits a
technology whose behavior changes faster than any premarket review can study it. For functions
that are low-acuity, reversible, and deployed under clinician supervision, competency
benchmarking plus an auditable monitoring plan should be sufficient premarket evidence, with
clinical confirmation generated in supervised operation after authorization. The premarket
question becomes whether the sponsor can demonstrate that the device is competent on the
simulation track and that the monitoring program will detect and respond to underperformance;
the clinical evidence then accumulates on real patients, under supervision, and gates the
device's progression.

The conditions that justify this are the conditions of the monitoring program itself. Performance
floors should be prespecified per function, numeric, and agreed with FDA before volume
accumulates. Sponsors should be permitted to deploy in phases of increasing autonomy, for
example: real-time supervision, in which a clinician reviews every output before it reaches the
patient; near-real-time asynchronous review, in which the device responds and a clinician
reviews within a bounded window, with actions such as prescription fulfillment held until review
completes and the patient notified of any correction; and autonomous operation with
sub-sampled asynchronous review, reached only when prespecified volume and floors have
been met. Progression between phases should require a submission; regression should be
automatic when a floor is breached. Breaches should be reported to FDA within a defined period,
and sustained underperformance should trigger reversion to the earlier phase without waiting
for agency action. The monitoring infrastructure should be auditable: operated under a quality
management system, with write-once audit trails, with sampling definitions and denominators
documented, and with aggregate safety and concordance reporting to FDA at a cadence agreed
in the authorization. One reporting design we would regard as acceptable, offered as an example
rather than the only option, is a quarterly performance report per function in the envelope
covering performance against each prespecified floor, breach and reversion events, escalation
sensitivity and over-escalation, judge-physician agreement on the sampled review, and the
volume and provenance of the sample, together with notification of any never-event to FDA
within 72 hours of the sponsor becoming aware of it. Sponsor attestation is not enough; the
records behind the report should be inspectable.
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

This approach is not appropriate where harm can occur before monitoring can detect it:
functions that take irreversible action without human review, functions deployed in time-critical
settings, or functions whose failure mode is silent (a missed diagnosis that manifests months
later). For those, premarket clinical evidence remains necessary. It is also not appropriate where
the sponsor cannot show that its monitoring works: a monitoring plan that has never detected
anything is not evidence of safety.

Q19. Postmarket performance evaluation
Comment on periodic re-benchmarking, sample-based clinician review and degradation
monitoring. What else? How should cadence and triggers be set?

We believe that all three approaches are potentially necessary depending on the exact device.
Periodic re-benchmarking should run against every release candidate and every
foundation-model change, not on a calendar; the trigger is the change. Sample-based clinician
review should be continuous in supervised phases (where it is the primary confirmation
mechanism) and sampled at a prespecified rate in autonomous phases, with the sample
enriched for cases the supervisory judges flag. Degradation monitoring should be continuous
and judge-based, with a numeric floor per function; a breach triggers reversion (Q18). To make
"numeric floor" concrete, take the emergency-recognition function of a primary-care device. Its
floor might be stated as: sensitivity to clinician-adjudicated emergencies in the sampled review
of not less than 95% on a rolling 90-day window, with the lower bound of the 95% confidence
interval not below 90%; a single clinician-confirmed missed emergency with potential for severe
harm reported to FDA within 72 hours regardless of the aggregate; and a monitored ceiling on
the proportion of escalations judged unnecessary, reported but not gating. A breach of the
sensitivity floor reverts the function to real-time supervision until two consecutive windows are
back above it. The figures are illustrative; the point is that each floor names the measure, the
sample, the window, the statistical bound, the response and the exit condition.

Two additions. First, shadow operation: before a candidate version takes over, it should run in
parallel with the deployed version on the same live cases, with its outputs scored but not shown
to patients, and should demonstrate non-inferiority on the prespecified floors before cutover.
During the shadow run the deployed version's decisions continue to govern the patient's care.
This matters for conversational devices, where a candidate may want to ask a question the
deployed version did not ask: the candidate's divergence is recorded and scored (including
whether its proposed question would have been clinically appropriate), but the conversation
follows the deployed version, so the assessment never changes what the patient experiences.
Where a candidate's divergence cannot be scored without the information it would have
gathered, that case is counted as unresolved rather than as a pass. Second, trajectory-level
monitoring for conversational devices (Q5): sampled production conversations scored for
conformance to the declared envelope, not only per-turn correctness.

Triggering events for reassessment should include any change to the model, prompts, retrieval
content or clinical protocols; any floor breach; any incident meeting a reporting threshold; and
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

any material change in the population served. Cadence should follow the rate of change rather
than the calendar: a device that ships weekly should be re-benchmarked weekly.

Q20. Machine-based supervisory agents
Can postmarket monitoring be facilitated by supervisory agents? What about evaluating and
ensuring the reliability of the supervisory agent itself?

Yes, and in practice there is no other way to monitor a device operating at conversational volume.
Supervisory judges can score every interaction for missed red flags, departures from guideline,
scope drift and escalation correctness, and route flagged cases to clinician review. The
considerations we would ask CDRH to adopt are these.

The supervisory agent should be treated as part of the device for regulatory purposes and
described in the submission: validated, versioned, change-controlled and covered by the same
regression testing as the device it monitors. For a device whose premarket evidence is reduced
in favor of postmarket monitoring (Q18), the supervisor is the mechanism that makes that trade
safe, so its reliability is central to the safety case rather than incidental to it. Its reliability should
be established against independent clinician adjudication on a held-out sample, with agreement
rates reported per task and re-established whenever the judge changes. In our own validation,
reported in the same manuscript under review, a judge classifying whether a physician edit
changed clinical meaning agreed with the majority of three blinded board-certified physicians
88% of the time (kappa 0.64, sensitivity 0.88, specificity 0.90), within the range of agreement
between the physicians themselves (kappa 0.60 to 0.70). Agreement was markedly lower for the
downstream judgments of appropriateness and counterfactual harm, among physicians as
much as between judge and physician, which is a reason to treat those endpoints as needing an
explicit calibrated reference rather than assuming a supervisory agent can be validated against a
single clinician. Where the judge is an LLM, it should be built on a different model family from
the device, so that a shared blind spot is not scored as a pass. Not every supervisory function
needs a model: deterministic checks (did the device recommend a dose, did it name a condition
outside the envelope, did an escalation phrase appear without an escalation action) should be
used wherever possible, because they are cheaper, auditable and do not drift. Sampling
definitions and denominators for everything the supervisor reports should be documented so
that the figures reaching FDA are interpretable. And the supervisor's own failure modes should
be monitored: a judge whose flag rate falls to zero is more likely broken than vindicated.

Q21. Stakeholder roles
What roles for clinicians, institutions, societies and standards bodies in postmarket monitoring,
without diffusing manufacturer accountability?

Manufacturer accountability should remain singular. Every other stakeholder role should be
structured as an input to the manufacturer's monitoring obligation rather than as a substitute for
it. Clinicians using or supervising the device are the primary source of adjudication and should
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

have a direct, low-friction channel for flagging outputs, with the manufacturer obliged to review
and report on those flags. Professional societies are the right owners of the clinical standards
against which outputs are graded; the role of societies, standards bodies and licensing
organizations in independent benchmarking is set out in our answer to Q16. Healthcare
institutions deploying the device should contribute outcome data and can audit the
manufacturer's monitoring, but should not be the sole monitors: institutions may be reluctant to
run evaluations that expose their own practice variation, and an institution-led monitoring model
would be uneven across sites. None of these roles should dilute the requirement that the
manufacturer detect, report and act on underperformance.

Section VI — Change control

Q22. Scaling re-evaluation to the modification
How might re-benchmarking scale to the nature and impact of a modification? Which changes
could be managed within QMS versus requiring premarket review? Which belong in a PCCP?

Re-evaluation should scale to the expected impact of the change, and we propose three tiers.
Changes managed within the quality management system: prompt refinements,
retrieval-content updates, user-interface changes and bug fixes. These should go live after
passing a fixed, pre-agreed regression suite (the simulation track) and a shadow run against the
deployed version on live cases, with the results retained for inspection. Changes covered by a
PCCP: a foundation-model version change within the same model family, tuning of thresholds
within a declared range, and additions to the competency envelope's case bank. These should
require full re-benchmarking, shadow operation against an acceptance criterion prespecified
per use case (for example, a non-inferiority margin on each monitored floor), and a notification to
FDA with the results, but not prior authorization. Changes requiring premarket review: changes
to the core decision logic or architecture, a new function added to the competency envelope, a
change of model family, and advancement of any function to a less supervised deployment
phase.

The QMS mechanisms that make the first tier safe are ordinary software controls applied to
clinical content: two-party review of every prompt and protocol change, continuous-integration
gating on the regression suite, and version control that lets any production output be traced to
the exact model, prompt and protocol that produced it.

Q23. PCCP when changes cannot be prespecified
How might PCCP concepts adapt when the nature or scope of future modifications cannot be
fully prespecified?

The PCCP should specify the test, not the change. For a GenAI device the set of future
modifications cannot be enumerated, but the capabilities the device must retain can be. We
propose that a GenAI PCCP consist of a capability floor: a fixed benchmark set covering every
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

function in the competency envelope, with a numeric floor per function; the production
monitoring floors from the postmarket plan; and a shadow-operation protocol with an
acceptance criterion prespecified per use case, such as a non-inferiority margin on each floor.
Any modification that passes all three is within the plan, whatever its nature. Any modification
that fails any of them is outside the plan and reverts.

This approach has the property CDRH needs: it is indifferent to the cause of a change, so a
prompt edit, a retrieval update and a vendor model upgrade are all evaluated the same way. The
benchmark set should itself be allowed to grow under the plan, as incidents and production
sampling surface new cases, with FDA notified of additions rather than approving each one; a
plan whose test set is frozen will fall behind the device it governs. This latitude is only appropriate
where the postmarket surveillance behind it is rigorous: auditable records of what was added
and why, floors that cannot be relaxed by adding cases, and the reporting and reversion
obligations described in Q18. Without those, a self-extending test set is a way to move the
goalposts, and CDRH would be right to refuse it.

We see a place for two kinds of GenAI device, and we expect them to coexist within one system
of devices. Narrow, modular devices with structured inputs and outputs and a rigorous,
indication-specific evaluation set are the right form for the most safety-critical functions, such as
emergency recognition or dose calculation, where the behavior must be fully specified and
testable in isolation. Broad, competency-based devices are needed for the breadth of ordinary
primary care and, in particular, for the safe management of patients with several conditions at
once, where a collection of single-indication modules cannot reason across the interactions
between them. A test-defined PCCP is what makes the second kind governable: a narrow device
can be re-evaluated against its one indication, but a broad device can only be re-evaluated
against a capability set, and a capability floor with production monitoring is the instrument for
that. A system that combines both, with the broad device orchestrating and the narrow devices
gating its safety-critical actions, should be able to hold both kinds of authorization at once.

Q24. Third-party foundation model changes
For devices built on third-party foundation models, how can a manufacturer detect, evaluate and
respond to developer-initiated changes? What contractual, technical or PCCP mechanisms give
reasonable assurance?

Foundation-model providers retain the right to make changes behind a pinned version, and a
manufacturer should assume that a model it did not train can change without notice. Reasonable
assurance therefore has to come from the manufacturer's own detection and response, not from
the provider's disclosures. Three mechanisms are available.

Technical: a canary benchmark: a fixed set of cases executed against the production model on a
continuous schedule, with results compared to a baseline. A statistically significant shift triggers
the same re-evaluation as a manufacturer-initiated change (Q22, Q23). Zero-retention,
no-fine-tuning configurations help ensure that the model evaluated is the model in production.
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Contractual: notification commitments for behavioral changes, version-retirement notice
periods long enough to re-evaluate and migrate, and the right to pin a version for a defined
period. PCCP: the test-defined plan in Q23 treats a vendor change as any other change, so the
response is predefined and does not require a new submission unless a floor is breached.

We would ask CDRH to encourage, through the MAF program (Q25) or otherwise, notification
and version-stability commitments from model developers, while recognizing that advance
notice will not always be possible or practical: a developer may need to push an urgent
safety-related update faster than any notice period allows, and a commitment should be framed
as notice where feasible and prompt disclosure after the fact where not. That is a further reason
the manufacturer's obligation to detect and respond must stand regardless of what the
developer discloses. We would also not want this answer read as a claim that detection is always
enough. For some applications, where the consequence of an undetected behavioral change is
severe and the other controls (shadow operation, supervisory judges, human review) cannot
bound it, a fully frozen model under the manufacturer's own control, with weights and inference
environment pinned and no provider-side updates possible, may be the only way to give
adequate assurance, and a manufacturer should be prepared to accept that constraint for those
functions.

Section VII — Other topics

Q25. Foundation model MAFs
Would voluntary Foundation Model MAFs be practical and kept current? What should they
contain? What would make a voluntary program useful given weak developer incentives?
Alternatives?

As a manufacturer building on third-party models, we would find Foundation Model MAFs
useful, and we think they are practical to provide if their content is kept to what the developer
already knows. The content we would ideally want, as a minimum, is: a stable version identifier
and a change log; the training-data cutoff; the safety evaluations the developer has run, with
results by subgroup where available; known limitations and failure modes in clinical use; and the
developer's commitments on notification of behavioral change and on version retirement. Of
these, the notification commitment is the one with the most regulatory value, because it is what
lets a manufacturer's change-control plan work (Q24).

Voluntary disclosure will be uneven, because developers have limited incentive and some liability
concern, and we recognize that some of the largest model developers may decline to participate
in a master-file program at all. The design of the device framework should therefore not depend
on MAFs existing. Two things would make the program useful nonetheless. First, a filed MAF
should not necessarily be a condition of authorization; a manufacturer should generally be able
to obtain authorization on the strength of its own evaluation and monitoring whether or not the
developer has filed, since a hard requirement would hand model developers an effective veto
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

over device makers. Second, the program should be framed so that filing is in the developer's
commercial interest: a developer whose model carries a current MAF is a developer whose
model is easier for sponsors to build on. The alternative mechanism, which we think should apply
regardless, is to require that sponsors demonstrate their own ability to detect and respond to
model changes. That makes the device safe irrespective of what the developer discloses.

Q26. Agentic devices
What additional considerations apply to agentic devices? How should the elevated risk of
autonomous multi-step action, tool use and reduced human review be reflected in acceptance
criteria and oversight?

Agentic devices differ in one respect that should drive everything else: the gap between an
incorrect output and a real-world consequence can close without a human in between. We
propose that acceptance criteria and oversight for agentic devices be built around that gap.

First, every action with a real-world effect should be classified in advance as one of three kinds:
logged only (reversible and low consequence, such as drafting a message), held for a defined
window during which a human can cancel (such as queuing a prescription for fulfillment), or
gated by explicit human approval before execution (such as any action that is irreversible or that
the envelope marks as high consequence). The device's intended use should state which actions
fall in which class, and conformance should be monitored in production.

Second, autonomy should be earned per function and should be revocable. A device can plan
autonomously for a function while its actions remain gated; the two should be assessed
separately. The deployment phases we describe in Q18 map onto clinical training: in the first
phase the device acts like a final-year student, with every output reviewed in real time; in the
second it acts like a senior resident, responding to the patient directly while a clinician reviews
the exchange asynchronously within a short, enforced window and can correct the response
before any action completes. We would propose that the window for this phase be on the order
of ten minutes rather than hours, which experience in deployed asynchronous telehealth
suggests is achievable at scale; that any downstream action with real-world effect, such as
transmitting a prescription, be held until the review completes; that the patient be told when a
response has been reviewed and notified of any correction; and that the review itself be logged
as a paired observation so that this phase continues to generate the evidence described in Q11. In
the third phase it acts autonomously, with sampled retrospective review. The device's
decision-making should be identical across phases, so that the absence of real-time supervision
does not change its behavior. Advancement requires evidence against prespecified floors; a floor
breach reverts the function to the prior phase automatically.

Third, escalation sensitivity should be evaluated in isolation, as its own function with its own floor,
because in an agentic device it is the control that bounds every other function. A device that is
excellent at the routine case and misses the emergency is unsafe, and that failure mode is
invisible in aggregate performance.
 Counsel Health | Comments on FDA-2026-N-7874 | October 19, 2026

Fourth, the supervisory agents that monitor an agentic device (Q20) should themselves be
gated from action: a monitor that can intervene is an agent, and should be evaluated as one.