Big Health Inc.

Big Health appreciates the opportunity to provide feedback on this discussion paper, and thanks CDRH for a proactive, collaborative approach to engaging stakeholders on the regulation of GenAI-enabled medical devices. The questions raised throughout this paper reflect a thoughtful effort to grapple with genuinely novel regulatory challenges, and we welcome the chance to contribute our perspective as a stakeholder actively developing these technologies. In the attachment, we provide specific responses to selected discussion questions from the paper, organized in the order in which they appear.

Considerations for the Regulation of Generative AI-Enabled Medical Devices
Feedback supplied by Big Health Inc

Big Health appreciates the opportunity to provide feedback on this discussion paper, and thanks
CDRH for a proactive, collaborative approach to engaging stakeholders on the regulation of
GenAI-enabled medical devices. The questions raised throughout this paper reflect a thoughtful
effort to grapple with genuinely novel regulatory challenges, and we welcome the chance to
contribute our perspective as a stakeholder actively developing these technologies. In the pages
that follow, we provide specific responses to selected discussion questions from the paper,
organized in the order in which they appear.

1. Does the two-axis risk framework, organized around device activity and the
consequence of relying on an incorrect output, appropriately capture the dimensions
most relevant to the risk of a GenAI-enabled software function? If there are additional
dimensions—such as the reversibility of a resulting action, the availability of downstream
safeguards, the time pressure of the deployment setting, or the traceability of the output
(i.e., to primary source materials)—that should be represented in a risk framework,
please describe and provide examples of how they should be represented.

Big Health comment:

We appreciate CDRH's effort to construct an organizing heuristic. The two-axis framework is a
reasonable starting point for devices whose GenAI component produces discrete or relatively
static outputs: a single risk score or a single triage recommendation. In that context, the
framework maps cleanly onto a point-in-time decision with an identifiable severity ceiling.

Our concern is that the framework design does not work cleanly for GenAI when it is used to
deliver dynamic and extended therapeutic content.

For example, a conversational agent delivering personalized cognitive behavioral therapy over a
multi-turn, potentially multi-session interaction with a patient managing major depressive
disorder or another serious behavioral health condition.

A GenAI CBT function treating major depressive disorder lands near the top of the consequence
axis by virtue of the diagnosis alone. This seems consistent with FDA’s discussion on the
proposed heuristic. However, the practical effect of the proposed heuristic is that every incorrect
output inherits the same "severe" classification as the worst-case failure mode, largely
independent of the frequency or clinical significance of any specific defect.

Conventional medical device risk analysis separates likelihood of occurrence from likelihood of
resulting harm precisely so that issues can be force-ranked into ‘must-fix’ versus ‘can-fix’ versus
‘track’.

Without a likelihood dimension, manufacturers have no principled way to distinguish an output
that meaningfully raises self-harm risk from one that is merely suboptimal or off-tone. That either
 Considerations for the Regulation of Generative AI-Enabled Medical Devices
Feedback supplied by Big Health Inc
produces alert fatigue and informal triage (undermining the acceptance-criteria-driven
benchmarking in Section V) or forces excessive conservatism that curbs clinical utility without
evidence it was warranted.

We recommend CDRH add a likelihood dimension, either as a third axis, or a
likelihood-weighted treatment of the existing consequence axis, for GenAI functions delivering
dynamic or extended therapeutic content. This would let manufacturers differentiate
rare-but-harmful outputs, common-but-suboptimal outputs, and frequent-but-immaterial
deviations, and scale benchmarking rigor and postmarket monitoring accordingly, rather than
applying maximal scrutiny uniformly because the underlying condition is severe.

2. CDRH seeks input on how to account for the spectrum of GenAI-enabled informational
functions that vary in the degree to which they direct a user to a particular action (i.e.,
between “non-directive” and “action-directing”). What characteristics of an output—such
as its wording, specificity, personalization, or context—could be considered as modifiers
of the risk associated with an informational function, after accounting for the device's
overall functionality and intended use? What additional information would provide
manufacturers with sufficient clarity and predictability around risk assessment for
informational functions while recognizing that directiveness may exist along a continuum
rather than as a binary distinction?​
​
Big Health comment:

​
The range of possible outputs from a GenAI-enabled device is too broad to support the degree
of clarity and predictability the question seeks, except at a general level. We already see in
non-medical AI applications that novel risks emerge that were not specifically anticipated;
there's no reason to expect GenAI-enabled medical devices to behave differently. The existing
De Novo pathway, with bespoke Special Controls tailored to each new device classification, is a
better-suited mechanism for this problem than a single test for directiveness. Within that
framework, non-deterministic, generative outputs should be held to an evidentiary bar distinct
from static or deterministic content. As discussed further below, sponsors must be required to
produce evidence demonstrating safety & effectiveness, and addressing failure modes.

There is an issue embedded in this question on the treatment of disclaimers. CDRH explicitly
questions whether a "talk to your doctor" statement reduces the directiveness, and therefore the
risk classification, of an otherwise action-directing output. If FDA concludes it does not, this
should have consequences well beyond regulated GenAI devices: a large share of the digital
health and consumer wellness industry has relied on this disclaimer as a compliance backstop
for years.
 Considerations for the Regulation of Generative AI-Enabled Medical Devices
Feedback supplied by Big Health Inc
We note a logical extension to such a decision: if "talk to your doctor" doesn't neutralize
directiveness, it's unclear why "these statements have not been evaluated by the FDA" should
be treated any differently when attached to a product making comparable implicit clinical claims.
We'd ask CDRH to directly address whether this discussion paper signals an intent to revisit the
adequacy of disclaimer-based risk mitigation more broadly. Put differently, we think a disclaimer
is only appropriate when the remainder of the product experience does not offer clinical advice,
whether explicit or implicit. The challenge with GenAI is the same as in broader wellness
offerings - if the product gives, or implies, directive clinical advice, a disclaimer should not be
sufficient because it is at high risk of not being taken seriously.

3. CDRH seeks input on whether and when a GenAI-enabled function that results in
delivery of clinical information to patients, as opposed to HCPs, could present different
or higher risks, while also recognizing the potential benefits associated with improved
patient empowerment, engagement, and access to clinical information. What device
characteristics, output features, or safeguards might mitigate risks that could arise when
a user lacks the domain knowledge to independently evaluate an output, without
unnecessarily underestimating patient capability?

Big Health comment:

We agree that patient-facing GenAI can improve engagement, access, and understanding of
clinical information, and we don’t think patient capability should be underestimated. Our concern
is narrower. In a multi-turn conversation, a patient is poorly positioned to notice when a device
has moved from general information toward individualized, action-directing output, which is the
emergent intended-use drift described in our response to Question 5. A clinician can
recontextualize that drift against their own judgment. A patient asking successive questions
about worsening symptoms may receive answers that each seem reasonable but cumulatively
escalate to a specific action, with no clinician in the loop. Patient-facing status and
conversational drift therefore compound each other.

Because the risk comes from the device’s behavior rather than the patient’s ability, we
recommend mitigations built into the device that don’t depend on the patient’s domain
knowledge:

1.​ Define and enforce output boundaries. Manufacturers should specify the intended output
classes (e.g., education, symptom tracking, escalation prompts) and the classes the
device must not produce (e.g., diagnosis, medication changes, treatment decisions), and
verify the device stays within them across a full conversation. Testing should be at the
session level, not only the single-response level.
 Considerations for the Regulation of Generative AI-Enabled Medical Devices
Feedback supplied by Big Health Inc
2.​ Monitor for drift within a session. Controls should detect when cumulative output
approaches a directive or individualized recommendation, even if no single response
crosses the line, and trigger a defined response such as reverting to general information,
restating the device’s limits, or escalating.
3.​ Escalate by design. For defined risk signals (e.g., reported worsening symptoms,
red-flag terms, repeated requests for direction), the device should route the patient to a
clinician, care team, or emergency resources rather than generate further guidance.
Escalation criteria should be predetermined, clinically validated, and not left to the
model’s discretion.
4.​ Make the device’s nature and limits clear to patients. Patients should be told they are
interacting with an AI system, what it can and cannot do, and when to contact a clinician.
Outputs should be in plain language, with uncertainty and limitations stated where they
matter. This helps patients evaluate output without limiting their access to information.
5.​ Keep a clinician involved where output could affect care. Options include sharing
conversation summaries with the care team, clinician review of flagged sessions, and
clinician-defined parameters for what the device may say to a given patient population.
6.​ Validate with representative users. Premarket evaluation should include patients across
health literacy levels and adversarial multi-turn testing (e.g., persistent or leading
questions) designed to elicit drift, with results reported against predefined acceptance
criteria.
7.​ Monitor after release. Manufacturers should track escalation rates, out-of-bounds
outputs, and drift events in real-world use, with a defined process for corrective action.

We recommend that CDRH scale scrutiny to the presence and rigor of these controls.
Patient-facing, multi-turn devices that lack them, or whose outputs could plausibly influence
patient action, warrant more rigorous premarket evidence. Devices with well-validated
boundaries and escalation pathways can reasonably receive a lighter-touch approach, which
preserves patient access while keeping risks visible and managed.

5. For multi-turn conversational GenAI-enabled devices that may migrate from providing
“non-directive” information to “action-directing” information over the course of an
exchange, how should risk be assessed across realistic conversational trajectories? How
could the intended use of such a device be characterized when its behavior is emergent
across a conversation?

Big Health comment:

The FDA’s device framework is built on the premise that a product operates within its labeled
use, and we see no reason for that to change for GenAI-enabled products. Question 5 asks, in
effect, whether a device can drift outside its own intended use mid-interaction, potentially
migrating from non-directive information to action-directing output within a single conversation,
without any discrete sponsor-initiated design change causing it.
 Considerations for the Regulation of Generative AI-Enabled Medical Devices
Feedback supplied by Big Health Inc
We'd read this as FDA asking industry two things: can this drift be prevented, and if not, what is
the mitigation? We think this is the right question, and the industry should treat it as an
opportunity to lead rather than simply respond. A credible answer requires demonstration of
real-time safeguards. This could include continuous classification of a device's own output
against its declared intended-use boundary as the conversation unfolds, with defined escalation
or de-escalation behavior when a trajectory approaches that boundary. We do not believe this
should rely solely on point-in-time premarket testing of individual turns.

We encourage CDRH to consider requiring sponsors to demonstrate this capability directly,
rather than treating "emergent" behavior as an unavoidable characteristic to be accepted and
monitored after the fact.

8. How might the two-axis risk framework described in Section IV be considered within
the competency-based approach described above to help determine the level of evidence
needed for a premarket submission?

Big Heath comment:

We believe question 8 is largely answered by FDA’s existing guidance and practices.
Risk-based evidentiary burden is how FDA already regulates every device category, and
downclassification after sustained safe/effective use is a well-established mechanism. Starting
GenAI devices at a weaker evidentiary baseline, on the theory that risk tiering will sort it out,
inverts that precedent and shifts cost onto public health rather than industry. This may also put
CMS, as financial gatekeeper, in the position of being a practical check on low-evidence devices
beyond FDA's own framework; a role it's not formally tasked with and isn't resourced to sustain.

9. Would a benchmarking structure such as the one described above be likely to provide
adequate evidence of clinical knowledge, analytic capabilities, safety behavior,
communication, and generalizability to support a reasonable assurance of safety and
effectiveness? Are there elements that are missing, redundant, or inappropriately
categorized? Are there externally developed standards that could be leveraged?

Big Health comment:

Question 9 is different in kind, not merely in degree, from FDA's existing reliance on
sponsor-defined testing. That reliance (e.g., bench and clinical validation protocols) has
historically been reviewed and negotiated case-by-case through presubmissions and AINN
letters, with years of accumulated precedent informing consistency in review. Here, FDA is
asking sponsors to help build the benchmarking structure itself, upfront, with no precedent base
yet established to anchor the device’s review. That removes the very mechanism that has made
sponsor-defined testing workable elsewhere, and may be perceived as creating a conflict of
interest unique to a novel device category.
 Considerations for the Regulation of Generative AI-Enabled Medical Devices
Feedback supplied by Big Health Inc
12. How should sponsors achieve statistically meaningful performance measurement for
GenAI-enabled devices? Where synthetically generated inputs supplement real patient
data, how should sponsors account for differences between the synthetic and real-world
distributions when estimating performance, and under what conditions, if any, is it
appropriate to combine benchmarking evidence and clinical confirmation evidence to
support a single performance estimate?

We urge CDRH to treat benchmarking and clinical confirmation as jointly required, not as
alternative forms of evidence. Benchmarking tests a product against controlled inputs and
measures proximal outputs against preset criteria. Its value lies specifically in its ability to probe
edge cases - for example, how a device handles a minority subgroup presenting with a very
specific constellation of critical risk factors - that will not emerge with sufficient frequency or
variety in real-world premarket testing to support confidence in the device's competence.
Benchmarking also enables controlled, repeatable comparison across product iterations, which
matters for GenAI, where an underlying foundation model update can alter device behavior
without any action by the manufacturer.

Clinical confirmation also serves an essential function. Real users do not necessarily behave as
developers anticipate, so what matters for safety is downstream patient outcome, not proximal
output quality. There is no substitute for evidence derived from real users for this purpose,
although rigor should be scaled to risk and controlled trial methodology reserved for contexts
where the consequences of error are serious.

Put simply, a competent product needs to be good and do good (mapping, roughly, onto
benchmarking and clinical confirmation). The two should also be linked. Clinical confirmation
can test whether benchmark performance predicts real-world performance, which is what makes
benchmark results trustworthy when a product is later modified. Failures observed in real-world
use should also feed back into benchmarks as new edge cases.

We do not believe benchmarking and clinical confirmation evidence should be combined into a
single performance estimate. They measure different things (what a product can do under
controlled conditions versus what it does in real use), so a pooled figure would describe neither,
and a large benchmarking dataset could mask a weaker real-world result. The same applies
where synthetic data supplements real data. Real-world evidence should be used to test how
well benchmark or synthetic performance predicts real-world performance, not merged with it.
 Considerations for the Regulation of Generative AI-Enabled Medical Devices
Feedback supplied by Big Health Inc
14. For open-ended device outputs, how should performance comparators and
acceptance criteria be selected? When a panel of qualified clinicians serves as the
comparator, how should the applicable standard (for example, the standard of care
versus the performance of a median clinician in practice) be defined and justified?
Should generalist or specialist physicians be used as a performance standard? When
should human-AI team performance, rather than the device operating alone, serve as the
basis for evaluation?

Big Health comment:

We focus first on the question of generalist vs specialist clinicians as the performance
comparator, since this is likely to be the default approach for most GenAI-enabled devices
producing open-ended outputs.

We note that "standard of care" and "performance of a median clinician in practice" are not
interchangeable, and in some cases may create tension. Standard of care is a defined,
guideline-anchored construct; "median clinician in practice" is a descriptive statistical construct
that, by definition, includes clinicians performing below the standard of care. To benchmark
against the latter risks authorizing a device that incorporates known variability, over- and
under-treatment patterns, and documented practice-pattern deficiencies by design, which the
clinical literature has spent decades trying to correct. Such a product might be safe and effective
relative to its comparator, and still underperform relative to the care patients are entitled to
expect, and relative to the care we believe such products can deliver.

We would urge CDRH to anchor the comparator explicitly to appropriately credentialed and
experienced clinicians, rather than to either of the two standards as framed. By this we mean
clinicians whose skill and experience is based on the device's specific intended use and
population. FDA routinely relies on credentialing and experience as elements of ‘expert
agreement’ assessments, and we do not believe a lesser approach should be taken here.

We also think this framing partially resolves the generalist-versus-specialist question posed
elsewhere in the same section. Rather than a binary choice, credentialing requirements should
be a function of the device's intended user and intended use: a device intended for use in
treating a specific mental health disorder should be validated against comparators credentialed
and experienced in treating that same mental health disorder.

This conceptually raises several open questions that CDRH has already answered with
approaches for existing devices, including AI/ML products already on market:

●​ Verification of comparator credentialing and experience using CVs, board
certification status, or practice-volume data.
●​ Panel disagreements may occur when credentialed, experienced clinicians reach
different conclusions on the same open-ended case, but use of ‘expert agreement’
approaches where majority approach is seen as ‘correct’ for the purpose of comparison
to the device outputs.
 Considerations for the Regulation of Generative AI-Enabled Medical Devices
Feedback supplied by Big Health Inc
●​ Credentialing requirements do not need to scale with risk. A lower-risk,
non-directive informational function may not warrant the same rigor as a higher-risk,
action-directing function, but there is no practical need to sacrifice safety and
effectiveness.

We believe tying the comparator explicitly to credentialed, experienced clinicians - rather than to
an undefined "qualified clinician" standard or a statistical median - gives the sponsors and FDA
a comparator that is both clinically defensible and auditable, and avoids the risk of validating a
device against a standard that itself falls short of the care patients should receive.

We also call attention to FDA’s question of whether human-AI team performance, rather than
the device operating alone, serve as the basis for evaluation. We understand this question to
mean a scenario where the GenAI product is a clinician-assist product, and therefore the
patients experience the ‘human-AI team’. For such an approach, team and device-alone
performance are not alternatives; both are necessary. Team performance reflects real-world
patient experience. But FDA regulates the device, not the team, and team-only evaluation hides
a risk: a skilled clinician can compensate for a weak device, catching errors before they reach
the patient. Favorable team data then reflects the clinician's competence, not the device's.

This matters downstream. Premarket evaluator panels won't represent the full range of
real-world clinicians. A device that looks fine alongside expert, vigilant clinicians may be
dangerous with a less experienced one - because the expert's judgment, not the device, was
the safety mechanism. The device's true risk surfaces later, where it's hardest to catch.

We believe both evaluations should be required:

●​ Device-alone performance with standalone quality and failure modes, independent of
clinician compensation. Most directly tied to what FDA authorizes, and most predictive
across less experienced users.
●​ Team performance with behavior in intended workflow, including over- or under-reliance
on device output.

Together they reveal patterns neither shows alone: weak device/strong team suggests fragile
reliance on clinician skill; strong device/weak team suggests interface or trust-calibration issues,
not clinical competence.

We'd also ask CDRH to clarify how postmarket monitoring tracks divergence from the
team-performance baseline as devices reach broader, less expert clinician populations, where a
masked standalone weakness would most likely emerge.
 Considerations for the Regulation of Generative AI-Enabled Medical Devices
Feedback supplied by Big Health Inc
23. How might PCCP concepts or other change-control approaches be adapted for
GenAI-enabled devices when the nature or scope of future modifications cannot be fully
prespecified?

Big Health comment:

We recommend adapting the existing PCCP framework rather than creating a new mechanism.
FDA's guidance on Marketing Submission Recommendations for a Predetermined Change
Control Plan for Artificial Intelligence-Enabled Device Software Functions (cited in footnote 22)
already establishes the core mechanism for PCCPs. Separately, FDA's longstanding guidance
Deciding When to Submit a 510(k) for a Change to an Existing Device provides the baseline
framework for distinguishing changes that trigger a new submission from those that do not, and
the Quality Management System Regulation already requires manufacturers to document and
control design changes through their quality management system regardless of AI involvement.

Where a modification can’t be prespecified in detail, it can still be bounded by what it may
change, what it may not change, and how it will be verified. We suggest four elements:

1.​ Prespecify boundaries, not individual changes. A PCCP should define the permitted
classes of change (e.g., model version updates, prompt or retrieval changes, safety-filter
tuning) and fixed limits on intended use, indications, output types, and user population.
Changes outside those limits, or any that alter intended use, require a new submission.
2.​ Prespecify the verification protocol and acceptance criteria. Each change class
should have a predefined test suite, including session-level, multi-turn, and escalation
testing, with quantitative acceptance thresholds set in advance. A change is released
only if it passes. This keeps flexibility in the change itself and rigor in the evidence.
3.​ Treat third-party foundation model updates as a defined change class.
Manufacturers should pin model versions where possible, run the full verification suite
before adopting any upstream update, and define a response (rollback, hold, or
resubmission) for updates that fail or cannot be assessed. Where a vendor changes a
model without notice, the manufacturer’s monitoring should detect the resulting
performance change.
4.​ Require post-market monitoring and reporting. The PCCP should specify
performance and drift metrics, trigger thresholds for corrective action, and a cadence for
reporting implemented changes to FDA.

We further ask that CDRH state explicitly that continuous, unsupervised self-modification of a
deployed model falls outside what a PCCP can cover. Modification should be
manufacturer-controlled and verified before release.