Auriva Health (Melinda B. Chu)

See attached file(s)The attached paper presents a new concept: Modular Synthetic Clinical Environment Qualification: Public Comment to FDA Docket FDA-2026-N-7874

This public comment proposes Modular Synthetic Clinical Environment Qualification (MSCEQ), a framework for independently qualifying reusable synthetic clinical environments before using them to evaluate generative AI-enabled medical devices. Environments would represent specific clinical settings, including changing patient states, multi-day trajectories, and interactions among patients and care teams. Independent clinical review and appropriate validation would assess each environment’s suitability for a defined context of use, while modular infrastructure would support adaptation across conditions and reduce duplicated development effort. Qualified environments could contribute to risk-proportionate evidence packages and support re-testing after device updates. The framework also emphasizes transparent disclosure of synthetic testing, retrospective human-data evaluation, and prospective human-patient evaluation, allowing healthcare institutions to establish their own evidence requirements for adoption. This proposal responds primarily to Questions 10, 13, and 16 of FDA Docket FDA-2026-N-7874, with related discussion of Questions 9, 11, 12, and 26.

Thank you for your consideration! Please do not hesitate to reach out for further discussion. We would be happy to lead or participate in development of this concept MSCEQ or any other new initiatives.

Melinda B. Chu, M.D. M.B.A.
CEO & Founder
Auriva Health
melinda@aurivahealth.com

Modular Synthetic Clinical Environment Qualification:
Public Comment to FDA Docket FDA-2026-N-7874
Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and Request
for Feedback
Melinda B. Chu, M.D., M.B.A.
Independent Researcher, Auriva Health

Primary responses to Questions 10, 13, and 16, with related responses to Questions 9, 11, 12, and 26.

This public comment proposes Modular Synthetic Clinical Environment Qualification (MSCEQ), a framework
for independently qualifying reusable synthetic clinical environments before using them to evaluate generative
AI-enabled medical devices. Environments would represent specific clinical settings, including changing
patient states, multi-day trajectories, and interactions among patients and care teams. Independent clinical
review and appropriate validation would assess each environment’s suitability for a defined context of use,
while modular infrastructure would support adaptation across conditions and reduce duplicated development
effort. Qualified environments could contribute to risk-proportionate evidence packages and support re-testing
after device updates. The framework also emphasizes transparent disclosure of synthetic testing, retrospective
human-data evaluation, and prospective human-patient evaluation, allowing healthcare institutions to establish
their own evidence requirements for adoption. This proposal responds primarily to Questions 10, 13, and 16 of
FDA Docket FDA-2026-N-7874, with related discussion of Questions 9, 11, 12, and 26.

Figure 1. MSCEQ qualification before device evaluation. Independent developers create a synthetic clinical
environment, which clinicians assess for a defined context of use before device evaluation. Results contribute to a riskproportionate evidence package with transparent disclosure of synthetic and human evaluation methods; device changes
prompt re-testing.

FDA-2026-N-7874 | 1
Modular Synthetic Clinical Environment Qualification
Modular Synthetic Clinical Environment Qualification (MSCEQ) is a proposed framework for independently
qualifying reusable synthetic clinical environments for defined contexts of use before using them to evaluate
AI-enabled medical devices. I support a competency-based approach combining structured benchmarking with
proportionate clinical confirmation. MSCEQ could strengthen this approach, particularly for devices with
open-ended, multi-turn, or agentic behavior.
These environments would be purpose-built simulations, including multi-agent and multi-day scenarios,
designed around specific intended uses and clinical settings. Examples include a one-week post-operative
recovery trajectory or a stroke rehabilitation pathway. They could represent changing patient states,
incomplete information, interactions among patients and care teams, and variations in user behavior.
Independent organizations or companies could develop and maintain the environments without requiring FDA
to build them itself.
Once a base environment and scoring framework exist, modular templates could support additional conditions
and workflows with less duplicated development effort. Each adaptation would still require qualification for its
own context of use. Shared infrastructure could reduce the cost of evaluation for multiple device developers
while preserving meaningful scrutiny of device performance.

Qualification before device evaluation
Before an environment is made available for sponsor evaluation, independent clinicians and, where
appropriate, other domain experts should interact with and stress-test it. Their review should assess clinical
plausibility, important edge cases, realism of multi-agent interactions, and artifacts that could distort testing.
Qualification should cover the environment and its scoring framework, with prespecified fidelity criteria,
acceptance criteria, and documentation of known limitations.
Clinical review can establish clinical relevance and content validity, but it cannot by itself demonstrate that
simulated performance predicts safe and effective real-world behavior. Comparison with available real-world
clinical trajectories and observed failure patterns should supply additional evidence of construct validity and
predictive relevance. Qualification should be limited to a defined context of use and reassessed after
significant updates. A post-operative recovery environment would not automatically be suitable for stroke
rehabilitation.

FDA-2026-N-7874 | 2
Responses to the discussion questions
Questions 10 and 9
An independently qualified environment designed around the intended use could provide more relevant
evidence than a generic public benchmark, particularly when performance depends on time progression,
incomplete data, and repeated interactions. Its predictive value would need to be demonstrated rather than
assumed. Evaluation should assess the competencies described in the paper, including clinical knowledge,
analysis, safety behavior, communication, and generalizability, using explicit scoring criteria and clinically
justified failure thresholds.
Independence before sponsor access is useful but insufficient to prevent later optimization to the test.
Providers should maintain sequestered evaluation scenarios, rotating challenge cases, version controls, and
documented access policies. Public documentation could describe the context of use and qualification methods
while protecting evaluation cases. Scenario provenance and checks for overlap with training or development
data would help address contamination risk.
Question 13
Synthetic environments are particularly useful for multi-day trajectories, rare but consequential presentations,
and interaction patterns that are difficult to sample in real data. Independent clinical stress-testing, comparison
with real-world observations, and explicit review of subgroup coverage could reduce the risk that simulations
reproduce the blind spots of the models being evaluated. Wherever feasible, scenario development should
draw on diverse clinical sources rather than relying solely on a model of the same class as the device.
Unresolved fidelity gaps should limit the claims supported by the evaluation.
Question 16
Developing, clinically qualifying, and maintaining these environments is a concrete role for independent third
parties. Independence criteria should include disclosure and management of conflicts, no financial stake in a
particular device passing, and separation of sponsor consulting from final assessment. Qualification methods,
contexts of use, and re-qualification requirements should be publicly documented. Transparent access terms,
multiple qualified providers, and proportionate fees would help avoid limiting competition or excluding
smaller developers.
CDRH could consider whether such environments and scoring frameworks may be suitable for qualification
through the Medical Device Development Tool program, where appropriate. This would require assessment
within a defined context of use and would not amount to blanket approval of a simulation platform or of
devices evaluated within it.
Questions 11 and 12
Qualified environments could contribute evidence within a risk-proportionate evaluation framework. Whether
that evidence is sufficient, and what additional clinical confirmation is needed, should depend on the intended
use, risk profile, and demonstrated relevance to anticipated real-world inputs. Synthetic testing should not
automatically be treated as equivalent to clinical confirmation. For higher-risk functions or poorly represented
clinical settings, real-patient evidence may remain necessary.
Performance estimates should report uncertainty and account for differences between synthetic and real-world
distributions. Repeated runs of one scenario should not be treated as independent patients. Synthetic and
human-data results should be reported separately unless there is a justified statistical basis for combining them.

FDA-2026-N-7874 | 3
Agentic evaluation and reuse after changes
In response to Question 26, multi-agent and multi-day environments could test autonomous sequences of
actions, tool use, delayed consequences, and opportunities for human review. Acceptance criteria should
address failure to escalate, unsafe actions, propagation of errors across steps, and adherence to limits on
autonomy. The environment should evaluate the complete device workflow, including safeguards and
interactions with users, rather than only the underlying model in isolation.
The same qualified environment could be re-run after model updates, prompt changes, or foundation-model
swaps, supporting assessment of changes against a premarket baseline and, where applicable, predetermined
change control plans. Reuse should preserve version histories and sequestered challenge sets. Results could
inform postmarket monitoring, but simulated re-testing would not replace observation of actual deployment
behavior. Changes to the environment itself should trigger reassessment when they affect its qualified context
of use or scoring validity.

Transparency of evidence and clinical adoption
I also encourage CDRH to consider clear disclosure of the types of evidence supporting regulatory
authorization. Where an authorized device has been evaluated in a synthetic environment, purchasers should
be able to identify that fact, understand the qualified context of use, and determine whether evaluation also
involved actual human patients or human patient data. FDA clearance or approval should not be presented as
implying prospective human clinical testing where none occurred.
Evidence descriptions should distinguish testing in an independently qualified synthetic clinical environment,
evaluation using retrospective human patient data, and prospective evaluation in human patients. These
methods can coexist in an evidence package, but they support different claims. Descriptions of testing should
accurately reflect the methods used and their limitations, alongside the specific regulatory authorization
granted.
Hospitals, clinicians, payers, and other purchasers should remain able to decide whether the available evidence
is sufficient for their deployment setting. A hospital could reasonably decline to purchase or deploy an FDAcleared or FDA-approved device until evidence from actual human patients is available. Regulatory
authorization and an institution’s purchasing decision serve different purposes. Transparent evidence
descriptions would preserve regulatory flexibility while allowing institutions to apply their own clinical
evidence requirements.

Recommendation to CDRH
I encourage CDRH to consider the MSCEQ framework and independently developed synthetic clinical
environments that are clinically qualified and appropriately validated for their defined contexts of use as
components of a competency-based evidence package. The environment should be independently assessed
first and then used to evaluate devices, with safeguards that preserve independence over time.
This approach would not eliminate the need for real-world evidence for higher-risk functions or solve every
synthetic-data challenge. It could provide a scalable path to more relevant and reproducible evaluation, reduce
duplicated infrastructure costs, and support proportionate evidence requirements, with transparent disclosure
of what was tested and what remains uncertain.

FDA-2026-N-7874 | 4
Figures illustrating the proposed MSCEQ framework

Figure 1. MSCEQ qualification before device evaluation. Independent developers create a synthetic clinical
environment, which clinicians assess for a defined context of use before device evaluation. Results contribute to a riskproportionate evidence package with transparent disclosure of synthetic and human evaluation methods; device changes
prompt re-testing.

Figure 2. Modular infrastructure within the MSCEQ framework. Reusable simulation infrastructure supports distinct
clinical modules, each requiring independent qualification and appropriate validation for its own context of use. Multiple
devices can use each qualified environment; qualification of one module does not automatically qualify another.

FDA-2026-N-7874 | 5