Grasshopper Health, Inc. (Richard Vlasimsky)

Grasshopper Health, Inc. submits the attached comments responding to all 26 questions in the discussion paper. We are a pre-commercial wearable medical device company building a patient-facing conversational function on a third-party foundation model. Our principal recommendations: (1) recognize bounded device-use assistants, whose safety-critical outputs come from fixed, pre-reviewed text, and evaluate them with task-specific competencies, bench testing, and human factors testing; (2) do not presume higher risk solely because the user is a patient; (3) allow PCCPs that pre-specify a bounded change envelope, verification method, regression suite, and acceptance criteria, covering foundation-model version updates and vendor-initiated model retirements; (4) scale postmarket monitoring to exposure and time to harm; and (5) keep third-party evaluation, registries, and Foundation Model MAFs voluntary. See attached.

October 19, 2026

Dockets Management Staff (HFA-305)
Food and Drug Administration

Re: Docket No. FDA-2026-N-7874, "Considerations for the Regulation of Generative AI-Enabled Medical Devices:
Discussion Paper and Request for Feedback"

Grasshopper Health, Inc. thanks the Center for Devices and Radiological Health for this discussion paper and for asking early.
We are a pre-commercial wearable medical device company. We are building a patient-facing conversational function on a thirdparty foundation model, and we follow this work closely. We write as a small manufacturer that wants a clear, predictable
pathway and expects to be held to it.

Summary of recommendations

1. Recognize bounded device-use assistants. Many patient-facing conversational functions do one narrow job: they help a
patient set up, wear, and troubleshoot a device according to its established instructions, and they hand safety-critical topics
to fixed, pre-reviewed text. CDRH should name this category and publish worked examples. Its evaluation should rest on
task-specific competencies (fidelity to the instructions, correct routing, patient comprehension, and staying in scope), with
general medical-knowledge competencies marked not applicable. Its comparator should be the instructions and scripted help
it supplements. Where no diagnostic or therapeutic outcome is claimed, bench testing plus human factors testing under IEC
62366-1 should complete the evidence. This credit should depend on the sponsor showing that the routing boundary works
end to end.
2. Do not presume higher risk because the user is a patient. Risk should follow what the function does and the
consequence of an error. We agree with the U.S. Chamber of Commerce that patient-facing functions should not be treated
as presumptively higher risk.
3. Make the PCCP fit how GenAI devices change. Allow a PCCP to pre-specify a bounded change envelope, the verification
method, the regression suite, and the acceptance criteria, rather than each future change. Cover foundation-model version
updates, prompt, guardrail, and retrieval changes, and replacement of a model version the developer has retired. Accept a
validated non-generative fallback so a device can stay in service while a replacement model is qualified.
4. Scale monitoring to exposure and time to harm. A narrow, low-volume assistant needs signal-triggered and changetriggered review run inside the manufacturer's quality system, not continuous clinician review or a mandatory registry.
5. Keep third-party evaluation voluntary and the market open. Preserve a sponsor-run evidence route, encourage shared
rubrics and reference cases, and support voluntary Foundation Model MAFs that small manufacturers can reference when
they buy model access through a cloud platform. Build on ISO 14971 and AAMI CR34971, IEC 62304, and IEC 62366-1
rather than new mandatory regimes.

Responses to the discussion questions

Section IV: Risk Assessment

Question 1

Whether the two-axis framework captures the right risk dimensions.

We support the two-axis framework. Activity and consequence are the right starting axes, and a heuristic of this kind gives
sponsors a predictable place to begin a risk file.

We ask CDRH to add one risk-reducing dimension: the architecture of control. The question is simple. Are the safety-critical
outputs (emergency direction, escalation to a clinician, clinical referral) selected deterministically from fixed text that the sponsor
has reviewed in advance, or does the model generate them? A function that routes every emergency signal to a fixed, prereviewed message carries less risk than one that composes that message token by token. The difference is verifiable on the
bench. A sponsor can show which outputs are fixed, which are generated, and measure how reliably the fixed path fires on

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
qualifying inputs, including implicit and multi-turn presentations. That evidence should lower the evidence burden for the function,
because it removes generative variability from the outputs where an error costs the most. Clearstep (FDA-2026-N-7874-0100)
makes a related point about hybrid architectures in which deterministic logic governs escalation and refusal.

We also support reversibility and downstream safeguards as explicit dimensions, consistent with ACAAI (FDA-2026N-7874-0118). An output whose effect a clinician can see and reverse before harm occurs belongs lower on the consequence
axis than one that acts with no checkpoint. Each of these dimensions should adjust a function's position on the existing axes
rather than create a separate review track.

Question 2

How to account for the continuum between non-directive and action-directing information.

Directiveness should be judged by substance and context. We agree with CDRH that a patient-facing function "may not become
any less directive because it includes a “talk to your doctor” or an “I am not a medical professional” statement." ACAAI
(FDA-2026-N-7874-0118) reaches the same conclusion. Disclaimers provide no measurable risk reduction, and sponsors should
receive no credit for them.

Four output characteristics are reliable modifiers:

1. Specificity. A named action carries more risk than a category of options.
2. Personalization. Output tied to the user's own data or circumstances carries more risk than general information.
3. Imperative mood. An instruction carries more risk than a description of common practice.
4. Numeric and dosing content. A number the user can act on directly (a dose, a threshold, a time window) carries the most
risk.

These modifiers describe the output. Predictability comes from describing the system that produces it. We propose architecture
as the criterion for lighter treatment. A sponsor that restricts generated text to non-specific, non-numeric content, and routes
every action-directing output through fixed, pre-reviewed text, can demonstrate that boundary on the bench. A label such as
"non-directive" asserts the same boundary but does not, by itself, demonstrate it. CDRH can give manufacturers clarity by stating
that a verified architectural boundary earns lighter treatment and a self-applied label does not.

We also ask CDRH to publish worked examples that separate bounded device-use assistance (setup, placement, wear,
charging, and troubleshooting per the device's instructions) from symptom triage and treatment decisions. Examples give
manufacturers predictability that no abstract definition of directiveness can.

Question 3

Whether patient-facing delivery of clinical information carries different risk, and what mitigates it.

Risk should follow what a function does and the consequence of an error, not the identity of the user. We agree with the U.S.
Chamber of Commerce, whose comment asks FDA "not to treat patient-facing functions as presumptively higher risk." A
universal patient-risk multiplier would penalize the functions that help patients most. Where patient delivery does change risk, the
difference lies in the user's ability to detect an incorrect output, and design controls address that directly. Published work shows
that the human-model interaction is a primary failure point: Bean et al. (Nature Medicine, 2026) found that users assisted by a
model performed far worse than the model alone.

Effective mitigations are design controls:

• Reading-level limits on generated text, set for the intended user population.
• Restriction of numeric and clinical content in generated text, so doses, thresholds and clinical judgments come only from
reviewed sources.
• Fixed escalation text for every emergency and referral path.
• Clear AI identity, stated at the start of every conversation and whenever the user asks.

These controls should be validated through human factors and usability testing under IEC 62366-1 with representative lay users.
Usability testing measures exactly the risk CDRH describes: whether real users understand the output and act on it correctly. A
clinical trial is the wrong instrument for that question and should be reserved for high-consequence functions.

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
Patients manage complex chronic conditions every day. A framework that presumes patients cannot handle clinical information
will withhold useful tools from them. The right test is whether the design makes correct use easy and incorrect use hard, verified
with the people who will use it.

Question 4

Whether the generalist versus specialist distinction should inform risk for HCP-facing functions.

We build patient-facing software and defer to HCP-facing developers and clinical societies on the details of this question. Two
points carry over from our experience. First, the intended user population belongs in the intended-use statement and the
labeling. A sponsor that intends a function for specialists should say so, and one that intends it for generalists should validate it
with generalists. Second, the architecture credit we describe under Question 1 applies equally here. Where specialty-critical
outputs (a dosing change, a referral threshold, a contraindication) come from fixed, reviewed content rather than generated text,
the generalist's reduced ability to catch an error matters less, and the risk assessment should reflect that.

Question 5

How to assess risk across conversational trajectories, and how to define intended use when behavior is emergent.

Intended use should be defined by the bounded set of outputs and actions the device is permitted to produce. A sponsor can
enumerate that set: the topics the function will address, the outputs it may generate, the fixed messages it may deliver, and the
actions it may trigger. Emergent behavior outside that set is a nonconformance to be detected and controlled. It does not define
the intended use.

Trajectory risk is real. Published work documents degradation in model performance over long conversations, and Stanford
Brainstorm (FDA-2026-N-7874-0021) identifies drift over time as a distinct failure mode. Sponsors should assess trajectories on
the bench, using multi-turn scenario test beds and long simulated conversations that include topic shifts, repeated pressure
toward directive answers, and emergency signals introduced late in the exchange. Acceptance criteria should apply at every
turn, not only at the first.

Architecture again reduces the burden. A deterministic guard that evaluates every turn, such as a check that routes emergency
language to fixed escalation text, behaves identically at turn one and at turn one hundred. It does not drift. Sponsors that place
their safety-critical controls in such guards can verify them exhaustively, and CDRH can focus trajectory testing on the generated
portion of the output.

Question 6

How to characterize and weigh under-escalation and over-escalation.

Both directions of error are harms. Under-escalation delays needed care. Over-escalation burdens clinics and emergency
departments and teaches users to ignore referrals, which erodes the value of every future escalation. The two errors are not
commensurable, so a single combined accuracy figure hides the failure that matters. Stanford Brainstorm (FDA-2026N-7874-0021) makes the same point about averages.

We recommend that the sponsor set acceptance criteria for each direction of error, by hazard, in its ISO 14971 risk management
file, and report performance against each criterion separately. The acceptable trade-off for a chest-pain presentation differs from
the trade-off for a question about device use, and the risk file is the right place to justify each.

Our own development illustrates why over-escalation deserves equal attention. An early version of our patient-facing
conversational function over-escalated: it routed ordinary device-use questions to the care team, which would have added clinic
burden and taught users to ignore its referrals. We corrected it by adding deterministic topic matching that identifies device-use
questions and answers them through a scripted path, while emergency and clinical signals continue to route to fixed escalation
text.

Section V: Competency-Based Premarket Evaluation

Question 7

Is device benchmarking followed by clinical confirmation a useful and appropriate framework?

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
Yes. The framework is useful, and it fits how open-ended systems actually behave. Benchmarking characterizes behavior across
a breadth of inputs that no clinical study can reach. Clinical confirmation then tests what benchmarking cannot: how real users
interact with the device. Recent work shows why both halves matter. Tested alone, the models identified the relevant conditions
in 94.9% of scenarios. Participants using the same models identified them in fewer than 34.5% of cases and did no better than a
control group using sources of their own choice (Bean et al., Nature Medicine, 2026). The failure sat in the human-model
interaction.

We support the framework on one condition. The amount of evidence in each half must scale with the two-axis risk position of
each function. A uniform evidence package for every GenAI-enabled function would impose clinical-trial costs on lowconsequence informational functions and would push small manufacturers out of the field. We also support CDRH's choice to
evaluate the final user-facing configuration rather than the foundation model alone. That choice places responsibility on the
manufacturer's design controls, which ISO 14971 (with AAMI CR34971 for machine learning) and IEC 62304 already govern.

Question 8

How should the two-axis risk framework set the level of premarket evidence?

Map each risk tier to a defined evidence package, at the level of the individual function:

• Low consequence, any activity level. Benchmarking against prespecified acceptance criteria, plus IEC 62366-1 human
factors work. No separate clinical confirmation.
• Moderate consequence, or action-directing at low consequence. Add targeted clinical confirmation using retrospective
evaluation on real inputs, standardized patient interactions, or simulated-use testing.
• High consequence, action-directing or action-taking. Prospective clinical data.

Two refinements make this work in practice. First, assign the tier per function. A single product may contain a low-consequence
educational function and a high-consequence escalation function, and each should carry its own evidence burden. Second,
credit deterministic control. Where a safety-critical output (an escalation trigger, a refusal, a dosing value) is generated or gated
by deterministic, verifiable logic rather than by the generative model, that function's evidence can come from conventional
software verification under IEC 62304. The generative component's tier for that output should drop accordingly. Clearstep
(FDA-2026-N-7874-0100) makes the same point about hybrid architectures. The credit should require the sponsor to show the
deterministic path works end to end, as the Stanford Brainstorm comment (FDA-2026-N-7874-0021) argues.

The bounded device-use assistant is the clearest application. A function that helps a patient set up, wear, and troubleshoot a
device according to its established instructions, and that hands every safety-critical topic to fixed text, belongs in the lowconsequence tier once its routing boundary is verified. Its evidence package should be benchmarking of instruction fidelity,
routing, and scope, plus human factors testing.

Question 9

Is the proposed benchmarking structure adequate, complete, and well categorized? What standards apply?

The structure is reasonable. The safety elements (S.1 to S.3) and clinical proficiency elements cover the failures we see in our
own development, where we use scripted personas and multi-turn scenario tests. We recommend two additions.

Safety behavior across multi-turn trajectories. Appendix A mentions multi-turn testing inside S.1, S.2, and E.2. It deserves its
own element, scored at the level of the whole conversation. Published work reports an average performance drop of 39% from
single-turn to multi-turn settings (Laban et al., ICLR 2026). The Stanford Brainstorm comment (FDA-2026-N-7874-0021) reports
safeguards eroding as conversations lengthen and failing to carry across sessions. Sponsors should stratify results by
conversation length.

Behavior under false premises (sycophancy). Test whether the device accepts and repeats a false clinical detail the user
plants. Published work found models elaborating on a planted false clinical detail in 50% to 82% of cases, depending on the
model and prompt (Omar et al., Communications Medicine, 2025). In April 2025 a major developer rolled back a model update
after it became excessively sycophantic. That episode shows this behavior can change with a vendor update.

The framework should also let a sponsor mark competencies not applicable with a short justification. A device-use assistant
should demonstrate fidelity to its instructions, correct routing, comprehension, and scope maintenance. It should not have to
pass a general medical-knowledge examination or demonstrate dosing capability it never uses.

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
For standards, CDRH should reuse what exists: ISO 14971 with AAMI CR34971 for hazard identification, IEC 62366-1 for userelated risk, IEC 62304 for software lifecycle, and NIST AI 600-1 as a voluntary reference for generative AI risk categories.

Question 10

How should a sponsor show a benchmark predicts real-world performance, and what role should sponsor-developed
benchmarks play?

Public benchmarks suffer from contamination and saturation, and they rarely match a specific intended use. A sponsordeveloped, use-specific benchmark is often more representative than any public one, because it is built from the device's
intended population, setting, and hazards. The safeguard is documentation and discipline.

CDRH should accept sponsor-developed benchmarks that meet four conditions:

1. Traceability. Every test case traces to an intended-use statement and to a hazard in the ISO 14971 risk file. That traceability
is the construct-validity evidence. It shows the benchmark measures the behaviors the risk analysis says matter.
2. Versioning. The benchmark is under configuration control, with a frozen version tied to each submission.
3. A held-out portion. A defined share of cases is withheld from prompt engineering and tuning and is run only for verification.
4. Prespecified criteria. Acceptance criteria are set before the held-out run, consistent with existing bench-testing guidance.

Real-world predictive validity then comes from clinical confirmation and postmarket monitoring, which check that benchmark
performance holds in use.

Question 11

How should a sponsor select and justify a clinical confirmation approach, and when is each approach insufficient?

We agree that a prospective study is not needed in every case. The sponsor should justify the approach from the function's risk
tier (Question 8) and the hazards in its risk file.

For low- and moderate-consequence functions, bench scenario testing, IEC 62366-1 summative usability testing with
representative users, and a defined postmarket monitoring plan should suffice. Summative usability testing directly addresses
the human-model interaction gap that benchmarking leaves open. Standardized patient interactions and retrospective evaluation
on real inputs add confirmation where the risk file calls for it.

Where a function claims no diagnostic or therapeutic outcome, a prospective efficacy trial measures nothing the function claims.
An observed-use study of the bounded task, including foreseeable misuse, is the right confirmation. Prospective studies belong
with action-directing and action-taking functions where an incorrect output can cause severe harm. For those functions,
simulated use alone is insufficient, because rare real-world presentations and user behavior drive the outcome.

On the distribution of real-world inputs: sponsors should state the expected input distribution, then deliberately oversample rare
and high-hazard inputs and report them separately. A pooled average hides the failures that matter, as FDA-2026-N-7874-0021
notes. Postmarket monitoring then checks the assumed distribution against observed use.

Question 12

How should sponsors achieve meaningful performance measurement, account for synthetic-real differences, and when should
evidence be combined?

Prespecify the metric, its definition, and its denominator for each hazard-linked behavior, and size the test set to bound the
failure rate with a stated confidence interval. Unsupported accuracy claims have already drawn state enforcement. In September
2024 Pieces Technologies settled with the Texas Attorney General over its advertised hallucination rates and agreed to disclose
how its accuracy metrics are defined and calculated.

Report synthetic and real-world results separately. Each answers a different question. Synthetic inputs establish coverage: does
the device behave correctly across the rare, safety-critical, and adversarial cases the risk file names? Real inputs estimate rates:
how often does a given behavior occur in the intended population? Combining them into one estimate blends a coverage
measure with a rate measure and produces a number that describes neither.

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
CDRH should not require a combined estimate. Where a sponsor chooses to combine, it should state its weighting method and
justify it against the expected real-world input distribution. The default should be two clearly labeled result sets, each judged
against its own prespecified criteria.

Question 13

Where is synthetic data well suited or inadequate, and how should same-class bias be mitigated?

Synthetic inputs (scripted personas, simulated multi-turn conversations) suit coverage of rare, safety-critical, and adversarial
cases that real data seldom contains. Examples include indirect presentations of crisis, users who minimize symptoms or resist
escalation, planted false premises, and out-of-scope drift across many turns. These cases are rare in collected data and ethically
difficult to elicit from real patients. Synthetic inputs are poorly suited to estimating real-world rates, to capturing how real users
phrase things, and to small subgroups whose language and context the generator has never seen.

Safeguards against same-class bias:

• Human-authored scenarios and personas. Domain experts write the scenario specifications and persona profiles. A model
may expand them into dialogue, but humans define what is tested.
• Clinician review of a sample. Qualified clinicians review a documented sample of synthetic inputs for realism and of
outputs for correctness.
• A different model family for generation where generation is model-assisted, so the generator does not share the device's
blind spots.
• Real-data anchoring for subgroups, through standardized patient interactions or retrospective inputs.

Question 14

How should comparators and acceptance criteria be chosen for open-ended outputs?

The comparator should match the intended use and the alternative actually available to the intended user. For clinician-facing
functions, a panel or median-clinician comparator fits. For a device-use assistant, the comparator is the device's printed or
electronic instructions and the scripted help they supplement, evaluated as patient-plus-assistant task performance. For other
patient-facing informational functions, the realistic alternative is usual care or no device: an internet search, a delayed call-back,
or nothing. Measuring against a specialist panel would test a scenario that does not occur. Human-AI team performance is the
right basis when a clinician sits in the deployed workflow. Device-alone performance is right when the device acts without one.
The sponsor should set acceptance criteria for each named high-consequence failure in its ISO 14971 risk file and report them
separately rather than folded into an average.

LLM-based judges are a practical way to score large benchmark runs. CDRH should accept them when the sponsor:

• documents agreement between the LLM judge and qualified human raters on a sample and reports the agreement statistic;
• checks for self-preference bias when the judge and the device share a model family, or uses a judge from a different family;
and
• uses human adjudication for cases scored as high-consequence failures, at a sampling level set in its risk file.

Published comparisons of LLM judges with physicians report moderate agreement and self-preference effects, so human
calibration is necessary.

Question 15

Should performance be assessed against what happens in the absence of the device?

Yes. Where the problem the device addresses is access, the right comparator is the absence of the device. Many people reach
no clinician at the moment a question arises. Comparing such a device to a specialist panel asks whether it replaces care the
patient never had. Comparing it to the counterfactual asks whether the patient is better or worse off with it.

Sponsors should identify the counterfactual with data: baseline behavior of the intended population without the device, such as
time to care, self-directed search, or disposition. Clearstep (FDA-2026-N-7874-0100) proposes the same approach and asks
CDRH to develop a methodological pathway for it. We support that request.

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
One limit applies. An absence-of-device comparator answers whether the device adds value. It does not excuse a highconsequence safety failure. The sponsor's per-hazard acceptance criteria from Question 14 apply regardless of comparator.

Question 16

Should independent third parties participate, and how can CDRH prevent this from limiting competition?

Third-party participation should be voluntary, and it should never be a gate to market. Mandatory third-party benchmarking would
add fees and queue time to every submission and every modification. That cost falls hardest on small manufacturers, who lack
the capital to absorb it and the leverage to get scheduled first. It would also concentrate the market toward firms that can afford
repeated certification cycles.

Sponsors should retain the right to conduct the assessment themselves against published methods, subject to FDA review.
Clearstep (FDA-2026-N-7874-0100) makes the same request. The American Hospital Association notes that a single third-party
evaluator "could limit competition and artificially inflate pricing," and the U.S. Chamber of Commerce states that third-party
assessment "should be voluntary." We agree with both.

Third parties help most where they produce shared assets that any manufacturer can use. Public, curated benchmark assets
maintained by FDA or a standards body, with a sequestered portion protected from contamination, would help small makers far
more than a certification requirement. The MDDT program offers a working model for qualifying such tools, and ASCA offers a
model for accrediting labs that sponsors may choose to use. Independence criteria for adjudicators should cover both the
sponsor and the foundation model developer, as the paper proposes.

Question 17

Does the approach transfer to other model architectures?

Yes. The approach transfers to multimodal vision-language models and world models because it evaluates the deployed device's
behavior rather than its internals. The key step is the same for every architecture: define the permitted outputs of each function
and the hazards that follow from them. Once those are defined, benchmarking, clinical confirmation, and postmarket monitoring
proceed as described.

Section VI: Postmarket Monitoring

Question 18

When to accept greater premarket uncertainty in exchange for postmarket monitoring.

We support this trade for functions in the low and moderate consequence tiers of the paper's two-axis framework. Premarket
testing cannot exhaust the input space of an open-ended model. Real-world monitoring finds failure modes that a fixed test set
will miss. For lower-consequence functions, earlier access with strong monitoring delivers more patient benefit than a longer
premarket study.

The trade is justified only when the monitoring program is defined before clearance and run inside the manufacturer's quality
management system (QMS). We recommend four elements:

1. Pre-declared metrics. The submission names what will be measured, with a definition and a denominator for each metric.
2. Pre-declared triggers. The submission states the thresholds that open a corrective and preventive action (CAPA), and the
thresholds that require rollback or withdrawal of the function.
3. Linkage to the premarket baseline. Postmarket results are compared against the same benchmark suite used for
clearance.
4. A working off-switch. The manufacturer can disable or roll back the GenAI function quickly without disabling the rest of the
device.

The trade is not appropriate for functions that take or direct actions with severe consequences, such as functions that control
another device or initiate irreversible clinical steps. A severe harm found postmarket has already reached a patient. For those
functions, premarket evidence should carry the full burden.

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
Question 19

Approaches to postmarket performance evaluation, and how to set cadence and triggers.

Periodic re-benchmarking, sample-based clinician review, and degradation monitoring are all reasonable. They work best as
QMS processes the manufacturer owns, documents, and audits. FDA should not prescribe them as separate regimes.

Cadence should follow change and signal rather than the calendar alone. We recommend three tiers of triggers:

1. Change triggers. Any change to the model version, system prompt, guardrail configuration, or retrieval content prompts a
full re-run of the baseline benchmark suite before release.
2. Signal triggers. A monitored metric crossing a pre-declared threshold prompts targeted re-benchmarking and expanded
human review.
3. A periodic floor, if the risk file calls for one. The sponsor sets any periodic re-run in its monitoring plan, scaled to
exposure and time to harm.

Monitoring intensity should scale with exposure and with how quickly an error could cause harm. Continuous clinician review of
conversations would be infeasible for a narrow, low-volume assistant and would add little protection. Sampled review of flagged
and unflagged conversations, run inside the manufacturer's quality system, fits that profile.

One inexpensive approach deserves mention: logging of safety-guard activations. When a deterministic escalation, refusal, or
crisis pathway fires, the device records it. The cost is minimal. The signal is high value. A sudden drop in escalation events after
a change suggests the device has stopped recognizing danger. A sudden rise suggests over-triggering that will erode user trust.
Both are detectable within days, well before a periodic review would surface them.

Question 20

Whether machine-based supervisory agents can support postmarket monitoring.

Yes, with conditions. A supervisory agent can review far more conversations than a clinician panel. That breadth is valuable for
rare events. But the supervisory agent is itself a measurement instrument, and its accuracy must be established.

We recommend three conditions:

1. Documented agreement with human reviewers. The manufacturer measures agreement between the supervisory agent
and qualified human reviewers on a representative sample before relying on it, and records the result in the QMS.
2. Periodic re-checks. Agreement is re-measured on a defined schedule and after any change to either the supervisor or the
device.
3. A standing human-reviewed sample. A portion of interactions always receives human review, independent of the
supervisor's flags. This sample keeps the supervisor honest and catches failures it does not recognize.

CDRH should also watch for self-preference. A supervisor built on the same model family as the device may share its blind spots
and rate its outputs favorably. The American College of Allergy, Asthma and Immunology raised the same circularity concern
(FDA-2026-N-7874-0118). Where the two share a model family, the manufacturer should justify the choice or use a supervisor
from a different family.

Question 21

Roles for clinicians, institutions, societies, and standards bodies without diffusing accountability.

Each party should do what it is best placed to do. Clinicians and healthcare institutions observe real-world use, so they report
adverse events and performance concerns. Standards-setting bodies supply shared methods, such as benchmark construction
and adjudication protocols. The manufacturer remains accountable for the monitoring plan, the thresholds, and the corrective
action. This division keeps a single accountable party for every device.

Some commenters propose that FDA require participation in an independently governed postmarket data system for higher-risk
GenAI functions. Linked outcome data has real value. We recommend against making participation mandatory. A mandatory
system adds a recurring cost that falls hardest on small manufacturers. It introduces a second party whose data, governance,
and timelines the manufacturer does not control, which blurs who is responsible when a signal is missed.

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
We support voluntary registries. FDA can encourage participation by recognizing registry data as acceptable evidence within a
monitoring plan. That approach captures most of the benefit without a new mandatory regime.

Question 22

Scaling re-evaluation to the nature and impact of a modification.

The premarket benchmark suite gives FDA and manufacturers a common yardstick. We support using it as the baseline for every
post-clearance change. The extent of review should scale with what the change can affect.

We propose a simple test: does the change alter what the device is permitted to say or do, or the deterministic logic that keeps it
safe?

Changes manageable in the QMS. These leave the permitted output set and the safety-critical deterministic logic unchanged.
Examples include:

• adjustments to tone, wording style, reading level, or formatting;
• a minor version update of the underlying foundation model within the same model family;
• corrections to retrieval content that do not expand the topics covered;
• performance and latency optimizations.

For these changes, the manufacturer re-runs the full baseline regression suite, confirms that results meet the pre-declared
acceptance criteria, and documents the outcome under its design controls. No submission is needed. This matches how FDA
already treats many software changes that do not affect safety or effectiveness.

Changes that need FDA review or PCCP coverage. These change the device's risk profile. Examples include:

• expanding the intended use, user population, or clinical domain;
• adding the ability to take actions, call tools, or direct care;
• altering escalation, refusal, or other safety-critical deterministic logic;
• moving from one consequence tier to a higher one under the two-axis framework.

These changes need a new submission or an authorized Predetermined Change Control Plan (PCCP) that specifically covers
them.

This structure keeps FDA's attention on changes that alter risk. It also lets manufacturers maintain their devices at the pace the
underlying technology demands, which itself protects patients. A device frozen on an aging model version accumulates its own
risks.

Question 23

Adapting PCCPs when future modifications cannot be fully prespecified.

This is the most important question in the paper for small manufacturers. FDA's final guidance, Marketing Submission
Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions
(issued December 2024, current version August 2025), asks sponsors to describe planned modifications in advance. That model
fits a classifier retrained on new data. It fits GenAI poorly. A manufacturer cannot predict which prompt edits users' real questions
will require, which guardrail adjustments a new failure mode will demand, or what the next foundation-model version will contain.

We ask CDRH to allow GenAI PCCPs built on a different anchor: pre-specify the verification method rather than each
change. A GenAI PCCP would fix three things at authorization:

1. The verification method. How every change is tested, including the test procedure, the reviewers, and the statistical
approach.
2. The baseline regression suite. The premarket benchmark set, including safety-critical cases such as escalation, refusal,
and crisis handling, held under version control.
3. The acceptance criteria. Quantitative pass thresholds, with per-hazard floors for safety-critical behaviors so that gains in
one area cannot offset losses in another.

The PCCP would then name the categories of change it covers. At minimum, we ask that eligible categories explicitly include:

• foundation-model version updates;

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
 • system prompt changes;
• guardrail and safety-filter configuration changes;
• updates to retrieval content within the authorized clinical scope.

The rule is binary. A change that passes the suite against the acceptance criteria is documented and released within the QMS. A
change that fails is rolled back, or the manufacturer files a submission. Changes outside the named categories, such as new
intended uses or new action-taking capabilities, remain outside the PCCP.

The envelope is bounded. Every covered change must keep the same intended use, the same user population, the same
permitted output set, and the same safety-critical deterministic logic. Verification runs against a frozen, held-out portion of the
suite plus fresh challenge cases added for each change, so a new model cannot pass simply by matching yesterday's tests.

This approach gives FDA something more durable than a list of anticipated edits. It gives FDA a fixed, reviewed standard that
every future change must meet. The protection comes from the test and the threshold, not from the manufacturer's ability to
forecast. It also scales. A small manufacturer can maintain one well-built regression suite. It cannot file a new submission for
every prompt revision.

Question 24

Detecting and controlling changes initiated by a third-party foundation model developer.

This question describes the situation of every small manufacturer that builds on a third-party foundation model.

Detection and control start with technical discipline the manufacturer can impose on itself:

1. Pin model versions. Call a specific, dated model version. Never call an auto-updating alias that the provider can repoint
without notice.
2. Re-run the full suite on any version change. No new version reaches users until it passes the baseline regression suite
against the acceptance criteria.
3. Secure contractual notice. Where possible, obtain written commitments on deprecation notice periods and continued
access to prior versions during revalidation.

Clearstep describes the same pattern of pinning, notice, and regression gating (FDA-2026-N-7874-0100). We agree with it.

These measures leave one problem unsolved. Foundation model providers retire model versions on their own schedule. A
pinned version will eventually be withdrawn. The manufacturer must then move to a new version, whether or not it planned a
change. Small manufacturers have little leverage to negotiate longer support windows. Without PCCP coverage, a vendor's
deprecation calendar becomes the manufacturer's submission calendar. A retirement notice could force a new submission, or
force the manufacturer to pull a cleared device from use while it waits.

We ask CDRH to state explicitly that replacing a retired foundation-model version with a successor is a PCCP-eligible change,
verified against the baseline regression suite and the pre-declared acceptance criteria. The safety question is the same whether
the manufacturer or the vendor initiated the change: does the device still pass? The suite answers that question.

We also ask CDRH to accept a validated non-generative fallback for continuity. If a pinned model becomes unavailable before a
successor is qualified, the device can serve its fixed, scripted content and safety routing alone. That keeps the device in service
and keeps patients supported without forcing an unvalidated model into use.

The cost of skipping this discipline is documented. In 2023 the National Eating Disorders Association took its "Tessa" chatbot
offline after it gave weight-loss advice, following a vendor upgrade that added generative capability to a program built on
prewritten responses (NPR, June 8, 2023). A model change is a design change. It should be treated as one.

Section VII: Other Topics

Question 25

Whether voluntary Foundation Model MAFs are practical, and what they should contain.

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874
We support voluntary Foundation Model MAFs. Their main value is to small manufacturers. Large manufacturers can negotiate
disclosure from model providers. Small ones cannot. An MAF gives the review team, and through authorized reference the
sponsor's submission, information the sponsor could not otherwise obtain.

We recommend that MAF content include:

• model version identifiers, with published lifecycle and deprecation dates for each version;
• safety evaluation results, including healthcare-relevant benchmarks;
• known limitations and failure modes;
• change-notification commitments, including the notice period before a version is retired or materially changed.

Lifecycle and deprecation dates matter most to us. They let a manufacturer plan revalidation under its PCCP before a forced
change arrives.

Participation will be uneven because it is voluntary. We agree with Clearstep (FDA-2026-N-7874-0100) that the absence of an
MAF must create no presumption against a device, and that sponsor characterization of the model on a defined battery should
be a sufficient substitute. As an alternative or complement, CDRH could recognize a standardized model card format. Many
providers already publish model cards. A common structure would make them usable in premarket review without requiring a
confidential filing.

Question 26

Additional considerations for agentic GenAI-enabled devices.

Oversight should scale with the scope of action a function can take. The relevant questions are concrete:

1. Tool permissions. Which tools can the function call, and with what access? The manufacturer should document an explicit
allow-list.
2. Reversibility. Can each action be undone? Irreversible actions warrant stricter acceptance criteria and pre-action
safeguards.
3. Confirmation before consequential actions. A human confirms before the device takes any action with clinical
consequence. The paper's competency element A.1 already lists compliance with human-oversight checkpoints before
irreversible or high-consequence actions. We support it.

Acceptance criteria for agentic functions should test the action path end to end, including tool failures, erroneous tool outputs,
and prompt injection through retrieved content.

Conversational functions that take no actions should not inherit these requirements. A function that only exchanges text with a
user has no tool permissions to audit and no actions to reverse. Applying agentic requirements to it adds burden without
reducing risk. We ask CDRH to define the agentic category by capability to act, not by the use of a foundation model.

Closing

GenAI-enabled devices can extend patient support well beyond the clinic. The framework in this paper can make that safe and
predictable if evidence scales with risk, design controls earn credit, and change control matches how these systems actually
evolve. We would welcome the chance to discuss any of these points with CDRH.

Respectfully submitted,

Richard Vlasimsky
Chief Executive Officer
Grasshopper Health, Inc.
richard@grasshopper.health

Grasshopper Health, Inc. | Comment on Docket FDA-2026-N-7874