Flo Health (Andras Meczner)

Comment from Flo

15 October 2026

Submitted electronically via Regulations.gov

Re: Docket No. FDA-2026-N-7874, Considerations for the Regulation of Generative
AI-Enabled Medical Devices: Discussion Paper and Request for Feedback (the “Paper”)

Dear Sir or Madam,

Flo Health (“Flo”) welcomes the opportunity to comment on the Paper. Flo develops consumer
digital wellness products focused on women’s health, and works closely within the regulatory
frameworks that govern digital health, including those that apply to software as a medical
device. Our comments therefore reflect experience on both sides of the device boundary that
the Paper discusses.
Our comments draw on a Flo-funded preprint evaluating a consumer wellness and general
health-information assistant [1]. The assistant is not marketed as a medical device. Diagnosis,
treatment recommendations and individualized triage were outside the studied function’s
intended scope. We voluntarily apply ISO 14971-informed risk management, content safety
evaluation and monitoring to it [1]. In that study, we evaluated 25 large language model (LLM)
judges using 12,193 real-world and synthetic question-answer pairs. Our recommendations on
device evidence requirements concern functions that meet the device definition; the study is
offered as methodological experience from evaluation of informational software, not as device
evidence.

1. The two-axis risk framework (Q1, Q3)

We support the two-axis risk framework. It would be helpful if the FDA explained how each axis
relates to risk. The activity axis describes how independently a function directs or takes action.
This can affect whether an error is caught before it causes harm, and therefore the probability
that a hazardous situation leads to harm (P2) (ISO 14971:2019, Annex C). However, the activity
category alone does not tell us that probability. The consequences axis should describe the
severity of the harm.

The Paper suggests that patient-facing and measurement functions could be shifted higher on
the consequences axis because users may not be able to evaluate the output themselves (pp.
7–8). In our view, this reason mainly concerns likelihood. A patient may be less able than a
clinician to catch an error, making a wrong answer more likely to be acted on. That does not, by
itself, mean that the resulting harm would be more severe.

Flo Health — Docket No. FDA-2026-N-7874 — Page 1
This distinction matters for Q3 because safeguards can change the chance that an error leads
to harm. Clear sourcing, scope limits and effective supervision may reduce that chance, with
evidence needed to show that they work. Concerns about users’ ability to recognize errors
should therefore be tested through comprehension and safeguard performance. Where the
assessment assumes human oversight, the evidence should show that it works in practice,
including whether automation bias undermines it.

The way a function is used can also change the possible harm. For example, an autonomous
system might repeat an incorrect insulin dose before anyone notices. Repeated dosing or
delayed care could increase severity as well as likelihood. The framework should allow for both
effects and make clear why each adjustment is made.

We respectfully suggest that the FDA:

1.​ Retain the activity categories and explain how they inform the assessment of likelihood
and severity.
2.​ Explain whether each modifier affects likelihood, severity or both. The modifiers
considered should include the intended user and their ability to check the output,
professional supervision, downstream safeguards, time pressure, traceability and
reversibility.
3.​ Allow the assessment to reflect safeguards built into the function, with evidence showing
that they work in the intended setting. A disclaimer alone should not justify a lower risk
assessment for an otherwise action-directing output.

2. Assessing the complete device (Q5, Q22, Q24)
We welcome FDA’s emphasis on evaluating the final configured device (p. 11). In our
experience, safety depends as much on the system around the model as on the model itself: its
prompts, retrieval, guardrails and interface. The same model can be safe in one setup and
unsafe in another. This system view should guide testing and change control, not only
premarket evaluation.

We respectfully suggest that the FDA:

1.​ Expect conversational devices to be tested across realistic multi-turn exchanges,
including attempts to push them beyond their intended scope, since risk can build up
over a conversation (Q5).
2.​ After a change, accept a documented change-impact assessment to determine the
retesting required for affected behaviors and interactions, while retaining integrated
checks of safety-critical controls (Q22).
3.​ Expect manufacturers to track model versions, detect upstream changes, including
unannounced ones, and be able to roll back to a validated configuration (Q24).

3. Staged real-world deployment (Q8, Q10, Q11, Q12, Q18)

For functions within device scope, we think it is important that GenAI products reach users in
stages: first a small, defined population, with closer monitoring and prespecified criteria before
exposure widens. Benchmarks, simulations and standardized patients find many problems, but
not all of them. For example, models are often evaluated with complete, well-structured inputs,
while real users typically provide incomplete context (Q10, Q11). Shadow deployment cannot

Flo Health — Docket No. FDA-2026-N-7874 — Page 2
close this gap for conversational devices, because users’ follow-up questions and actions
depend on the outputs they see.

Before marketing authorization, staged exposure can be conducted as a clinical investigation
under 21 CFR Part 812, with stages and expansion criteria prespecified in the investigational
plan. Greater reliance on postmarket evidence (Q18) may be justified where predeployment
evidence supports initial exposure, problems can be detected promptly and corrective action
can prevent serious or irreversible harm. Such reliance is unsuitable where serious or
irreversible harm could occur before detection and intervention, or where important clinical
consequences cannot be reliably observed.

We respectfully suggest that the FDA:

1.​ Recognize staged deployment as a confirmation approach, and identify lower-risk uses
where benchmarking plus staged confirmation can be sufficient (Q8, Q11).
2.​ Expect prespecified stage sizes, follow-up periods and expansion criteria, with justified
sample sizes and confidence bounds for hazard-specific failure rates, and sampling that
reflects intended users and relevant subgroups (Q12).
3.​ Expect monitoring to be designed with data minimization, de-identification and strict
access controls, consistent with applicable privacy law.
4.​ Expect serious incidents to be reviewed before any expansion, and defined criteria for
pausing, restricting or rolling back deployment. The absence of new failure types should
count as supporting evidence, not as proof that rare harms are controlled.
5.​ Clarify how a staged investigational plan could transition into a post-authorization staged
rollout.

4. Machine-based supervisory agents (Q20)
We support using supervisory agents, under human oversight, to monitor GenAI-enabled
devices. Human review alone does not scale to real-world volumes, and it is less consistent
than often assumed. Even experienced clinicians can disagree about whether a response is
safe; in the study we cite, agreement between adjudicators was only moderate [1]. Agents that
each apply an explicit criterion can make evaluation more auditable and link findings to hazards,
provided the accuracy and reproducibility of their judgments are demonstrated.
In that study, dedicated judges reached 95.3% recall and 61.1% precision for failures against an
adjudicated reference standard. Calibration on real-world cases raised macro-F1 from 72.9% to
86.7%, and judge-derived ordinal risk estimates matched those derived from the reference
standard for all four in-scope hazards [1]. These results come from content evaluation of an
informational assistant. They show the approach is feasible; they do not show performance in
device applications or effects on clinical outcomes.
Where an LLM acts as an adjudicator in benchmarking, it should meet the same structural
independence criteria as human adjudicators (p. 14); validated judges operated by an
independent third party could support this. FDA could build on existing expectations for
quality-system software by clarifying how they apply to probabilistic evaluators. These include
21 CFR Part 820 (QMSR), which incorporates ISO 13485:2016 including clause 4.1.6, and
FDA’s software assurance guidance [2].
We respectfully suggest that the FDA:
1.​ Develop guidance on validating probabilistic evaluators, using representative, held-out
outputs of the monitored device and an independently adjudicated reference standard.
Recall and precision should be reported for each criterion, with uncertainty estimates

Flo Health — Docket No. FDA-2026-N-7874 — Page 3
 and justified acceptance thresholds. Recall for serious failures should take priority, but
the human-review workload created by false positives should also be assessed.
2.​ Expect checks of reproducibility, subgroup performance and shared blind spots between
the agent and the device, especially when they use related models. Expect revalidation
after changes to the device, the criteria or the agent’s own model.
3.​ Expect continued human review of samples of both flagged and passed cases, so that
missed failures and new failure types are still found.
4.​ Consider the Medical Device Development Tool program as a route to qualifying agents
for a defined context of use.
Flo supports proportionate evidence requirements that protect patients and enable access to
well-controlled generative AI health tools. We welcome the opportunity to discuss these
comments with CDRH.
Sincerely,

Andras Meczner
Director of Medical Accuracy and Safety, Flo Health

References
[1] Meczner A, et al. Clinical Safety Evaluation of LLM Assistants Using Decomposed LLM-as-Judges: A Framework
Grounded in Real-World Interactions and Formal Risk Assessment. SSRN. 2026. Preprint, not peer reviewed.
https://doi.org/10.2139/ssrn.7230054
[2] US Food and Drug Administration. Computer Software Assurance for Production and Quality Management
System Software. Guidance for Industry and Food and Drug Administration Staff. February 2026.
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/computer-software-assurance-production
-and-quality-management-system-software

Flo Health — Docket No. FDA-2026-N-7874 — Page 4