Mosaic Clinical Technologies (Nina Kottler)

Mosaic Clinical Technologies (Mosaic) appreciates the opportunity to share comments with the Food and Drug Administration’s (FDA) Center for Devices and Radiological Health (CDRH) regarding its discussion paper on considerations for the regulation of generative artificial intelligence (GenAI)-enabled medical devices (Docket No. FDA-2026-N-7874).

Mosaic, the technology services division of Radiology Partners (RadPartners), develops and deploys technology infrastructure for enterprise imaging, including MosaicOS, a cloud-native and AI-native imaging management platform. Mosaic also includes Cognita Imaging, Inc. (Cognita), its AI incubator division, which develops vision language models for radiology, including an FDA breakthrough designation for a chest radiograph report generation device. Cognita recently received an FDA BAA focusing on the use of generative AI-based evaluation of novel medical devices. Through RadPartners’ managed practices, Mosaic has operational visibility into AI implementation across a national radiology network operating in all 50 states and interpreting more than 50 million imaging studies annually.

Mosaic supports CDRH’s goal of enabling a nimble, risk-proportionate regulatory approach that applies least burdensome principles while providing reasonable assurance of safety and effectiveness. GenAI-enabled devices require an oversight model that recognizes both the limitations of premarket evaluation and the importance of real-world clinical performance. Premarket evaluation should establish an appropriate threshold for market authorization based on intended use and risk, while postmarket monitoring and clinical governance should provide continuing evidence that AI-enabled clinical systems remain safe and effective in practice.

Please see attached for full comments.

Respectfully Submitted,
Dr. Nina Kottler, MD, MS, FSIIM, FAIM, Chief Medical AI Officer, Mosaic Clinical Technologies
Dr. Anand Singh, MD, FACR, Associate Director, Mosaic Clinical Technologies
Dr. Akshay Chaudhari, PhD, Co-Founder Cognita Imaging (acquired by Mosaic Clinical Technologies) and Associate Professor of Radiology and Biomedical Data Science at Stanford University
Justine Jamero, MHA, Associate Director, Health Policy & Advocacy

October 19, 2026
Digital Health Center of Excellence (DHCoE)
Center for Devices and Radiological Health (CDRH)
Food & Drug Administration (FDA)
5630 Fishers Lane, Rm. 1061
Rockville, MD 20852

Re: Docket No. FDA-2026-N-7874: “Considerations for the Regulation of Generative AI-Enabled
Medical Devices: Discussion Paper and Request for Feedback”

Mosaic Clinical Technologies™ (Mosaic) appreciates the opportunity to share comments with the Food
and Drug Administration’s (FDA) Center for Devices and Radiological Health (CDRH) regarding its
discussion paper on considerations for the regulation of generative artificial intelligence (GenAI)-enabled
medical devices (Docket No. FDA-2026-N-7874).

Mosaic, the technology services division of Radiology Partners (RadPartners), develops and deploys
technology infrastructure for enterprise imaging, including MosaicOS™, a cloud-native and AI-native
imaging management platform. Mosaic also includes Cognita Imaging, Inc. (Cognita), its AI incubator
division, which develops vision language models for radiology, including an FDA breakthrough designation
for a chest radiograph report generation device. Cognita recently received an FDA BAA focusing on the
use of generative AI-based evaluation of novel medical devices. Through RadPartners’ managed practices,
Mosaic has operational visibility into AI implementation across a national radiology network operating in
all 50 states and interpreting more than 50 million imaging studies annually.

Mosaic supports CDRH’s goal of enabling a nimble, risk-proportionate regulatory approach that applies
least burdensome principles while providing reasonable assurance of safety and effectiveness. GenAIenabled devices require an oversight model that recognizes both the limitations of premarket evaluation
and the importance of real-world clinical performance. Premarket evaluation should establish an
appropriate threshold for market authorization based on intended use and risk, while postmarket
monitoring and clinical governance should provide continuing evidence that AI-enabled clinical systems
remain safe and effective in practice.

Drawing on Mosaic’s experience deploying and monitoring AI across a national radiology network, our
comments focus on four principles:
1. Regulatory requirements should be proportionate to what the AI does and the consequences if it
is wrong.
2. Competency-based evaluation should assess the final user-facing device while preserving
meaningful clinical confirmation.
3. Postmarket monitoring should evaluate not only device performance but also the safety of the
broader human-AI clinical system; and
4. Monitoring must be scalable, actionable, and supported by appropriate infrastructure and clinical
resources.

The discussion paper raises a series of questions concerning risk assessment, premarket evaluation,
postmarket monitoring, and ongoing oversight of GenAI-enabled medical devices. Mosaic’s comments
will address selected questions in Sections IV-VII of the discussion paper.
Section IV: Considerations for the Assessment of Risk for GenAI-Enabled Devices

1. Does the two-axis risk framework, organized around device activity and the consequence of relying
on an incorrect output, appropriately capture the dimensions most relevant to the risk of a GenAIenabled software function? If there are additional dimensions—such as the reversibility of a
resulting action, the availability of downstream safeguards, the time pressure of the deployment
setting, or the traceability of the output (i.e., to primary source materials)—that should be
represented in a risk framework, please describe and provide examples of how they should be
represented.

Mosaic supports CDRH's proposed two-axis framework as a useful organizing heuristic. Regulatory
requirements should be proportionate to risk rather than applying identical evidentiary and oversight
requirements to every GenAI-enabled device. We agree that the degree and independence of device
activity and the potential consequences of relying on an incorrect output are foundational considerations.

The additional considerations identified by CDRH, including reversibility, downstream safeguards, time
pressure, and traceability, are also important because they affect whether an incorrect AI output is likely
to translate into clinical harm. We recommend treating these factors as contextual risk modifiers rather
than creating additional independent axes. The same device function may present materially different
real-world clinical risk depending on the ability of users to verify its output, the opportunity to intervene
before an action occurs, the expertise of the intended user, and the degree of transparency and
reviewability provided within the clinical workflow.

For example, in medical imaging, the American College of Radiology defines a qualified end-user as a
licensed practitioner with the qualifications and training to independently interpret the underlying
medical images without the aid of the AI and to assess the validity of the AI output. 1 All else being equal,
a radiology AI function used by such a qualified end-user may present a lower risk profile than the same
function used by an individual who cannot independently evaluate the underlying images or determine
whether the AI output is correct.

Human oversight should therefore not be treated as a binary safeguard. Its effectiveness depends on both
whether the intended user has the expertise necessary to independently evaluate the underlying clinical
information and on whether the device provides sufficient information to meaningfully assess its output
at the point of care. In medical imaging, risk may be lower when the AI provides clinically meaningful
information supporting independent review, such as the images or text used as inputs, relevant source
information, confidence or uncertainty information when appropriate, and visual localization, such as a
bounding box or heat map when available. Conversely, a device that provides only a conclusion, without
allowing the user to determine what information the AI considered or where an identified abnormality is
located, may increase inappropriate reliance or make errors more difficult to recognize.

CDRH should consider the availability and quality of point-of-care transparency and reviewability as a risk
modifier and should incentivize manufacturers to provide information that enables qualified end-users to
independently evaluate AI outputs within the clinical workflow. Transparency should be clinically
American College of Radiology and Society for Imaging Informatics in Medicine. ACR-SIIM Practice Parameter for Imaging Artificial Intelligence.
https://gravitas.acr.org/PPTS/GetDocumentView?docId=217.
meaningful and tailored to the function of the device rather than satisfied solely through static labeling
or technical documentation.

CDRH should apply these contextual factors to modify the risk assessment derived from the two-axis
framework. Factors such as reversibility, effective downstream safeguards, the qualifications and
independent evaluative capability of the intended user, time pressure, traceability, point-of-care
transparency and reviewability, human-AI interaction, escalation pathways, structured oversight
mechanisms and deployment environment may increase or decrease real-world clinical risk. The resulting
risk assessment should inform the type and amount of premarket evidence, appropriate safeguards and
labeling, and the intensity of postmarket monitoring.

Importantly, while FDA’s regulatory focus appropriately remains on the GenAI-enabled medical device,
the relevant unit of clinical safety after deployment may be broader. Real-world safety depends on the
interaction among the device, the clinician, the patient population, the workflow, and the local
deployment environment. A device may continue to meet its technical performance specifications while
unsafe patterns of human-AI interaction or use emerge. This distinction should inform postmarket
monitoring expectations.

2. CDRH seeks input on how to account for the spectrum of GenAI-enabled informational functions
that vary in the degree to which they direct a user to a particular action (i.e., between “nondirective” and “action-directing”). What characteristics of an output—such as its wording,
specificity, personalization, or context—could be considered as modifiers of the risk associated with
an informational function, after accounting for the device’s overall functionality and intended use?
What additional information would provide manufacturers with sufficient clarity and predictability
around risk assessment for informational functions while recognizing that directiveness may exist
along a continuum rather than as a binary distinction?

Mosaic agrees that directiveness exists along a continuum rather than as a binary distinction and should
be assessed based on the substance, specificity, personalization, certainty, and context of the output
rather than the presence or absence of particular words such as “recommend” or “should.” Importantly,
directiveness should be distinguished from overall risk. An output may be highly directive but relatively
low risk if the potential consequence of error is limited, while a less directive output may still present
substantial risk when users are likely to rely on it in a high-consequence setting.

Characteristics of an output that may impact risk associated with an informational function include:
• Specificity: Outputs that identify or endorse a specific course of action are more directive than
outputs that provide general information, describe alternatives, or present a range of options.
• Personalization: Outputs tailored to the individual patient's clinical circumstances are more likely
to influence action than generalized information and may therefore be more directive.
• Actionability, certainty and urgency: Outputs that present a specific action as definitive, urgent,
or time-sensitive are more likely to drive behavior than outputs that present alternatives or
appropriately communicate uncertainty.
• Presentation and workflow placement: Directiveness is influenced not only by the content of an
output but by how it is presented within the clinical workflow. An output that is prominently
displayed, pre-populated into a report, inserted into an order, or positioned as the default
recommendation may function as more directive than the same information presented as
 optional supporting information. CDRH should therefore consider the user interface and workflow
context when assessing practical directiveness.
• Intended user and independent evaluative capability: The practical effect of an informational
output depends on whether the intended user can independently evaluate its basis and
appropriateness. The same output may function as more directive when presented to a user who
lacks the domain expertise, source information, or workflow access needed to verify it. CDRH
should therefore consider not only who receives the output, but whether that user can
meaningfully challenge or override it.

Disclaimers should not materially reduce the assessed directiveness of an output when the substance of
the output continues to recommend or instruct a specific action. A statement such as “consult your
physician” should not offset otherwise highly personalized, specific, or action-directing clinical advice.

To provide manufacturers with greater clarity and predictability, CDRH should assess using illustrative
scenarios that combine content, personalization, certainty, workflow presentation, and intended-user
characteristics. Directiveness should not be determined by isolated words or disclaimers, nor should it
serve as a standalone risk classification. Instead, it should be treated as one component of risk assessment
and interpreted together with the consequences of relying on an incorrect output and the user’s ability
to independently evaluate or override the information.

3. CDRH seeks input on whether and when a GenAI-enabled function that results in delivery of clinical
information to patients, as opposed to HCPs, could present different or higher risks, while also
recognizing the potential benefits associated with improved patient empowerment, engagement,
and access to clinical information. What device characteristics, output features, or safeguards might
mitigate risks that could arise when a user lacks the domain knowledge to independently evaluate
an output, without unnecessarily underestimating patient capability?

Mosaic does not currently deploy patient-facing GenAI-enabled clinical functions and therefore offers
limited comment on this question. We agree, however, that patient-facing delivery should not itself
determine risk. Risk should depend on the nature and directiveness of the information, the consequences
of relying on an incorrect output, and the safeguards available to support appropriate interpretation and
escalation. In medical imaging, AI-generated information that independently interprets images, generates
new diagnostic conclusions, or recommends clinical action may warrant clinician review before
communication to patients.

4. CDRH seeks input on whether and how the distinction between generalist and specialist physicians
might inform the assessment of risk for HCP-facing GenAI-enabled functions. Under what
circumstances, if any, might risk be affected when an HCP who lacks the relevant clinical specialist
knowledge receives information that falls within a specialist area of practice? What device
characteristics or safeguards might mitigate such risks?

Mosaic believes the distinction between generalist and specialist physicians is a critical consideration
when assessing risk of HCP-facing GenAI-enabled functions, particularly when safe use depends on the
user’s ability to independently evaluate, challenge, and contextualize the AI output. For functions
involving diagnosis, treatment recommendations, classification, or management decisions, risk may
increase when outputs are presented to HCPs who lack the domain-specific expertise necessary to
determine whether the output is accurate, complete, and appropriate for the clinical context.
In medical imaging, this distinction is particularly relevant because interpretation of imaging findings
requires specialized training and integration of information that may extend beyond the AI output itself,
including a variety of prior exams, clinical history, and the significance of findings within the broader
diagnostic context. A qualified radiology end-user who can independently interpret the underlying images
may therefore provide an important downstream safeguard. The positive predictive value of an AI output
depends in part on disease prevalence, which may influence how results should be interpreted and acted
upon in different clinical settings. For uncommon findings, even a model with strong sensitivity and
specificity may generate a meaningful number of false-positive results that must be recognized and
appropriately managed in clinical practice.

The intended user’s expertise, training, and ability to appropriately act upon the clinical information
provided by the device are important considerations in assessing risk. For example, AI-enabled detection,
triage, and notification functions may provide valuable information to non-specialist physicians who can
recognize the significance of a potential finding and initiate appropriate clinical evaluation or escalation.
However, functions that support definitive diagnoses, treatment selection, or other complex clinical
decisions may require greater domain-specific expertise and the ability to independently evaluate AI
outputs. Specialist review by a qualified end user who can independently assess the veracity of AIgenerated findings may therefore serve as an important safeguard when warranted by the intended use
and associated clinical risks. 2

CDRH should therefore consider the intended user’s domain-specific expertise and independent
evaluative capability as risk modifiers. The intended-user assessment should consider not only who
receives the output but what level of expertise is necessary to safely interpret and act on it, particularly
in settings where low disease prevalence may reduce positive predictive value and increase the burden of
false-positive findings. Potential safeguards may include user-specific labeling, limitations on intended
users, specialist review for higher-risk functions, and point-of-care transparency that enables the intended
user to understand and independently evaluate the basis and limitations of the AI output.

5. For multi-turn conversational GenAI-enabled devices that may migrate from providing “nondirective” information to “action-directing” information over the course of an exchange, how
should risk be assessed across realistic conversational trajectories? How could the intended use of
such a device be characterized when its behavior is emergent across a conversation?

Mosaic does not currently develop or deploy multi-turn conversational GenAI-enabled medical devices
and therefore does not offer specific comment on this question.

6. For GenAI-enabled care escalation functions, CDRH is considering how the evaluation may account
for both under-escalation and over-escalation. How could manufacturers characterize and weigh
these two directions of error, given that they may not be commensurable and that acceptable tradeoffs may vary by clinical context?

Mosaic agrees that the relative risk of under-escalation and over-escalation will vary by intended use and
clinical context, and therefore a single weighting should not be applied across all GenAI-enabled functions.
Rather than creating a separate framework for these error types, manufacturers should characterize the

2 American College of Radiology and Society for Imaging Informatics in Medicine. ACR-SIIM Practice Parameter for Imaging Artificial Intelligence.

https://gravitas.acr.org/PPTS/GetDocumentView?docId=217.
anticipated consequences of under- and over-escalation within CDRH’s proposed two-axis risk framework,
considering both the degree and independence of device activity and the consequence of relying on an
incorrect output.

Because the real-world balance between under-escalation and over-escalation may differ from premarket
expectations, postmarket monitoring should assess these signals separately over time. Monitoring should
identify whether either error type is occurring at a clinically meaningful rate, whether the balance is
changing, and whether the resulting workflow or patient-safety impact warrants remediation. This
approach preserves a manageable risk framework while allowing oversight to adapt to how the device
actually performs in clinical practice.

Section V: A Competency-Based Approach for Premarket Evaluation of GenAI-Enabled Devices

7. Is the competency-based approach described above, i.e., device benchmarking followed by clinical
confirmation, a useful and appropriate framework for evaluating GenAI-enabled devices?

Mosaic supports the overall concept of a competency-based approach that combines scalable device
benchmarking with risk-proportionate clinical confirmation. This structure appropriately recognizes that
GenAI-enabled devices may have too broad a range of potential inputs and outputs for exhaustive
premarket testing, while preserving the need to demonstrate that the final user-facing device performs
safely and effectively under clinically representative conditions.

The value of benchmarking, however, will depend on whether the benchmark meaningfully evaluates the
competencies required for the device’s specific intended use. Benchmark performance should therefore
be interpreted as evidence of defined device capabilities rather than as a substitute for clinical
performance. Where benchmark assets do not adequately represent the intended patient population,
clinical task, or deployment context, clinical confirmation becomes particularly important. When clinical
judgment is required to establish the reference standard, adjudicate device outputs, or determine clinical
acceptability, that assessment should incorporate qualified clinical reviewers from the healthcare
organization deploying the device or a qualified clinical group engaged by that organization. Appropriate
safeguards, such as blinded adjudication, predefined evaluation criteria, and management of potential
conflicts of interest, should support objective clinical assessment.

The challenge with current benchmarks is that they are static and often publicly available. This gives rise
to a phenomenon, more recently coined in the broader frontier model development cycles, called
“benchmaxxing”. This occurs when a publicly available benchmark becomes the sign of progress and
competitiveness, leading to over-optimization of the particular benchmark, independent of real-world
model performance for tasks that the benchmark represents. Consequently, we advocate for benchmarks
to be dynamic so that they can continue providing value as human behavior and practice patterns change.
In doing so, dynamic benchmarking can minimize overfitting to a static set of questions. We believe that
such a concept of dynamic benchmarking fits within the total life cycle evaluation of devices due to its
similarity with continuous post-market monitoring.

Mosaic recommends a multi-stage risk-proportionate approach: use initial premarket benchmarking to
efficiently evaluate a broad range of relevant device capabilities, use clinical confirmation to determine
whether those capabilities translate into acceptable performance in clinically representative use, and
translate those findings into user education and expectation setting before deployment. This premarket
benchmarking can only allow assessment of the device performance. Findings from benchmarking and
clinical confirmation should also inform pre-deployment user education and expectation setting. Users
should understand the circumstances in which the device performs well, its known limitations and failure
modes, and the types of cases or outputs that warrant greater scrutiny.

Following deployment where humans interact with the device, it is critical to evaluate the human + device
performance. In this scenario, continued postmarket benchmarking that can incorporate both human and
device monitoring should be used to confirm that safety and effectiveness are maintained in practice.
Findings from benchmarking at this stage should inform whether premarket benchmarking mimics postmarket clinical use. While premarket benchmarking can offer practical guidance for intended users to help
mitigate over-reliance, complacency, and inappropriate distrust, postmarket benchmarking can measure
these tendencies in practice.

Overall, such a dynamic benchmarking approach that can maintain the rigor of each stage should be
proportionate to the risk characterized through CDRH’s two-axis framework.

8. How might the two-axis risk framework described in Section IV be considered within the
competency-based approach described above to help determine the level of evidence needed for a
premarket submission?

The level of evidence in premarket submission should be proportionate to the device's intended use and
risk profile. Devices associated with higher-risk activities should generally be supported by more robust
evidence during premarket submission. The two-axis framework should also inform the scope of
evaluation. For example, higher-risk devices may require assessment across a broader range of clinical
scenarios, more challenging test conditions, and more extensive clinical confirmation, accounting for the
implicit variability in the practice of medicine. In contrast, lower-risk devices may be supported by more
targeted evaluation.

9. Would a benchmarking structure such as the one described above be likely to provide adequate
evidence of clinical knowledge, analytic capabilities, safety behavior, communication, and
generalizability to support a reasonable assurance of safety and effectiveness? Are there elements
that are missing, redundant, or inappropriately categorized? Are there externally developed
standards that could be leveraged?

Mosaic believes these categories provide a reasonable organizing structure for contemporary GenAI
devices that have a clearly defined intended use and do not yet carry high risk. Benchmarking should
evaluate clinically relevant competencies rather than require every element for every device. As the level
of risk for future devices increases, Mosaic believes that it will be important to account for the implicit
human variability that exists in the practice of medicine. For example, for current regulatory paradigms
and the generation of ground truth, a 2+1 reader framework is typically used for a multitude of devices.
In this framework, if two physicians disagree with one another, a third adjudicator can provide a tiebreaking verdict. Here, the minority physician's verdict is considered incorrect and is superseded by the
majority verdict. However, depending on the intended use and different patterns in the practice of
medicine, there is a likelihood that both disagreeing opinions could be correct. Future benchmarking that
can incorporate this probabilistic variability in human decision making and the varying practice of medicine
will allow for better characterization of devices and their utility in the real world.
Where appropriate, externally developed specialty-society, consensus, or standards-based evaluation
approaches should be leveraged rather than creating duplicative requirements, provided their relevance
and validity for the device’s intended use are demonstrated.

10. CDRH recognizes that publicly available benchmarking assets may be subject to data contamination,
saturation, and limited real-world representativeness. How should a sponsor establish that
performance on a given benchmark predicts safe and effective real-world behavior for the device’s
intended use? What evidence should support the construct validity of a benchmark used to gate
device evaluation, and what role should sponsor-developed benchmarks play given potential
concerns around independence and optimization to the test?

Mosaic agrees that publicly available benchmarks may be affected by data contamination, saturation, and
limited representativeness of real-world clinical practice. As CDRH notes, benchmarking may not fully
capture interactions between users and workflows. For this reason, benchmarking should be viewed as
one source of evidence rather than a standalone predictor of real-world safety and effectiveness. As we
described previously, it is feasible to perform post-training of frontier models to align them to the specific
nuances of a given benchmark. Whether this comes at the expense of the intended use of the device is
unclear. This is the fundamental challenge behind benchmarking. We posit that since current
benchmarking only captures a static point in time, dynamic benchmarking that can be conducted in
prospective settings is necessary to capture the human-device interaction. Doing so can help establish
whether a device is safe and effective in the real world. Construct validity should be supported by evidence
that the benchmark measures competencies relevant to the intended clinical use and that performance
on those competencies meaningfully relates to performance under clinically representative conditions.

Sponsor-developed benchmarks may play an important role when existing benchmarks do not adequately
reflect a device's intended use. Sponsors should provide evidence supporting the relevance and validity
of those benchmarks and explain how they relate to real-world clinical performance. Clinical confirmation
remains essential to evaluate whether the deployed human-AI system performs safely and effectively in
practice. Benchmarks that are maintained internally at the FDA (firewalled from the outside world) or by
FDA-endorsed third parties may offer a pragmatic means of evaluating devices that have the same or
similar intended uses.

11. CDRH is considering that clinical confirmation for a GenAI-enabled device might not require a
prospective clinical study in every case and has described above a range of approaches of increasing
rigor and patient exposure. How might a sponsor select and justify a confirmation approach tailored
to a device’s intended use and proportionate to the device’s risk profile? Are there device types or
risk profiles for which one or more of these approaches would be insufficient or inappropriate? How
might the anticipated distribution of real-world inputs be taken into consideration? Are there other
methods of clinical confirmation that might help inform the evaluation of GenAI-enabled devices?

Mosaic supports a risk-proportionate, least-burdensome approach to clinical confirmation. For lower-risk,
assistive GenAI-enabled devices in which a qualified clinician independently reviews the relevant clinical
information and retains responsibility for the clinical decision, premarket clinical confirmation should
establish an appropriate threshold for authorization without attempting to reproduce every potential
deployment environment.
Some uncertainties are inherently local and may be better evaluated after authorization. Patient
populations, disease prevalence, technical environments, workflow integration, and clinician interaction
may differ substantially across deployment settings. Shadow deployment can help assess performance on
local real-world inputs and technical integration, but because device outputs are not presented to
clinicians, it cannot evaluate the safety of human-AI interaction. Where clinician interaction materially
affects performance, a controlled or limited clinical deployment with appropriate oversight and
monitoring may provide more meaningful evidence before broader scale-up.

This approach should not be applied uniformly across risk levels. Autonomous functions and higher-risk
devices, particularly those for which an incorrect output could lead directly to significant harm without an
effective opportunity for human review or intervention, should require more extensive clinical evidence
before authorization. Even for low-risk devices, user behavior may change based on the device’s
characteristics. In such situations, a retrospective-only evaluation may not truly depict the real-world
impact of the device. The amount of uncertainty that may appropriately be addressed through local
validation and monitored deployment should therefore be determined using CDRH’s two-axis risk
framework.

Manufacturers may sponsor and operationalize clinical confirmation studies, but the clinical evaluation of
device outputs should not rely solely on manufacturer-affiliated judgment. Where expert interpretation is
required, reference standards and adjudication should be established by qualified clinicians who are
independent of the manufacturer and, where applicable, the underlying foundation model developer. This
separation helps preserve the credibility of the clinical evidence without requiring an entirely separate
evaluation infrastructure.

12. How should sponsors achieve statistically meaningful performance measurement for GenAIenabled devices? Where synthetically generated inputs supplement real patient data, how should
sponsors account for differences between the synthetic and real-world distributions when
estimating performance, and under what conditions, if any, is it appropriate to combine
benchmarking evidence and clinical confirmation evidence to support a single performance
estimate?

Mosaic recommends that statistically meaningful performance measurement be based primarily on
representative real-patient data, with prespecified endpoints, acceptance criteria, and appropriate
assessment of uncertainty. Evaluation should reflect the intended-use population and include clinically
relevant subgroup analyses and sufficient representation of important failure modes.

Synthetic data can be used to enhance the training of most models. There are numerous examples of such
synthetic data pipelines being used to train both text and vision models across model types, including
bespoke task-specific models and larger frontier models. However, existing data simulators cannot yet
replicate the nuances of real clinical data, especially across multimodal settings for the purposes of
standalone evaluation. Thus, pivotal premarket and postmarket evaluation should primarily be conducted
on real data.

Given the evidence around the quality of current synthetic data, it may be useful as a supplemental tool
for stress testing, evaluating uncommon scenarios, or expanding coverage of conditions that are difficult
to represent adequately in available clinical datasets. However, synthetic data should not be assumed to
reproduce the clinical distribution, or latent relationships present in real patient data. It should only be
used in secondary settings to augment conclusions that are otherwise infeasible to make with real data
alone. Where synthetic inputs are used, their performance results should generally be reported separately
from results obtained using real patient data, and sponsors should justify how the synthetic data were
generated, what clinical characteristics they preserve, and what limitations may affect their applicability.
In the future, high-quality data simulators could be qualified using existing regulatory pathways, such as
that of the Medical Device Development Tools (MDDT).

Mosaic does not recommend routinely combining benchmarking evidence and clinical-confirmation
evidence into a single performance estimate. These evaluations may assess different aspects of
performance under different conditions and data distributions. Combining them could obscure meaningful
differences between controlled benchmark performance and performance under clinically representative
conditions. Results should generally be reported separately and interpreted together as complementary
evidence.

13. For which clinical domains, device functions, or subpopulations is synthetic data particularly wellsuited, or particularly inadequate, as a supplement to real-world evidence? What safeguards would
mitigate the risk that synthetic data generated by models of the same class as the device under
evaluation reproduces the very performance gaps the evaluation is intended to detect, particularly
for underrepresented subgroups?

While Mosaic cannot opine about the quality of synthetic data across all clinical domains, it does have
insights based on academic and applied work in the domains of synthetic text generation and radiological
image generation. Through these domains, generative text-only models have the capacity to generate
high-quality synthetic patient and clinical vignettes. Care must be taken to ensure that the real examples
used to seed or train synthetic data generators provide adequate coverage, so that the resulting datasets
are representative. If model optimization is used in this regard, the underlying reward model should be
robust to overfitting, which is a common failure mode. For synthetic text generation, modern frontier
models with broad knowledge may be able to extrapolate beyond their original training distribution.

Generating high-quality synthetic medical images is more challenging than generating text. Contemporary
models use approaches such as diffusion or rectified flow to generate visually pleasing images, replacing
prior methods that used generative adversarial networks. Typically, the best image generation is
performed using conditional generative models, where additional criteria (such as radiology reports,
clinical information, etc.) are provided at inference to guide the unbounded generative image pathway in
a particular direction in its learned manifold. Text-guided image generation is mature in the context of 2D
radiographic imaging, including extensive work that our team has performed previously. Newer
approaches are emerging for cross-sectional images, but both techniques do not have the requisite quality
today to serve as stand-ins that can replace the need for real data. Current image generation models can
only interpolate within their training distribution and are unlikely to extrapolate beyond it.

Mosaic believes synthetic data could have a limited role as a supplemental stress-testing tool but should
not serve as the primary basis for demonstrating clinical performance or generalizability. Its greatest value
may be in testing rare, high-severity scenarios, predefined edge cases, or controlled perturbations that are
difficult to capture at sufficient scale in real-world datasets. This approach requires documented
capabilities to recreate images that are rare or out of distribution, both of which are technically challenging
currently.
Synthetic data are particularly inadequate when the goal is to estimate real-world clinical performance
across representative patient populations or underrepresented subgroups. In these settings, synthetic
generation may reproduce or mask the same gaps, biases, and limitations present in the source data or
model class, creating false confidence in performance.

Where synthetic data are used, sponsors should clearly define the purpose of the synthetic dataset, report
those results separately from performance on real patient data and confirm clinically important findings
using independent real-world data whenever feasible. Synthetic data should not be used to substitute for
missing evidence in underrepresented populations unless the resulting findings are validated against real
patient data.

When synthetic data are generated by models like the device under evaluation, additional safeguards are
warranted. Sponsors should avoid relying solely on the same model family or training ecosystem to both
generate and evaluate test cases and should incorporate independent clinical review and real-world
confirmation of key findings.

14. For open-ended device outputs, how should performance comparators and acceptance criteria be
selected? When a panel of qualified clinicians serves as the comparator, how should the applicable
standard (for example, the standard of care versus the performance of a median clinician in
practice) be defined and justified? Should generalist or specialist physicians be used as a
performance standard? When should human-AI team performance, rather than the device
operating alone, serve as the basis for evaluation?

For open-ended outputs, the appropriate comparator and acceptance criteria should be defined based on
the device’s intended use, intended user, and clinical risk. Acceptance criteria should be prespecified,
clinically meaningful, and proportionate to the consequences of an incorrect or incomplete output. Where
a clinical standard of care can be reasonably established, it should serve as the reference standard.

The qualifications of the clinical comparator should reflect the device’s intended use and the specific
clinical performance being evaluated. Where safe use depends on independent verification of the
underlying clinical information, the intended user should have the domain-specific expertise necessary
to perform that verification without relying on the AI output itself. The qualifications of the person
establishing diagnostic accuracy should not necessarily be determined by the qualifications of the
intended user. An emergency physician may be the intended user of a triage alert, but an expert
radiologist may still be needed to establish whether the findings are present. Functions such as triage,
care coordination, screening, or referral should be evaluated in the context of the capabilities and
responsibilities of their intended users, which may differ from functions requiring specialist
interpretation. For intended uses with inherently higher inter-reader variability, regulatory evaluation
should account for this variability, rather than relying exclusively on a single adjudicated “correct”
outcome, which may not reflect the realities of clinical practice.

In radiology, this means evaluation by qualified physicians with expertise in interpreting medical imaging,
particularly when assessing diagnostic accuracy and the veracity of AI-generated findings. This expectation
does not preclude other clinicians from receiving AI-generated information for appropriate clinical actions,
including triage and escalation, consistent with the device’s intended use. As imaging becomes increasingly
shareable across institutions and geographies, access to specialist expertise should become less
constrained by hyperlocal capabilities, making it less appropriate to anchor performance expectations
solely to the expertise available at an individual site.

If a device is intended for use by individuals who lack the expertise to independently evaluate the
underlying clinical information, the evidentiary and performance expectations should be correspondingly
higher because the user provides less effective downstream error detection. In such settings, CDRH should
consider whether stronger clinical validation, higher performance requirements, additional safeguards, or
narrower intended-use limitations are necessary to maintain an acceptable risk profile.

For assistive GenAI-enabled devices, human-AI team performance may be the more clinically meaningful
measure of clinical performance because the intended clinical use depends on interaction between the AI
and a qualified clinician. AI-alone performance should still be characterized to understand the capabilities
and limitations of the device and to inform clinician education, but clinical evaluation should also assess
whether use of the device improves, maintains, or adversely affects the performance of the clinician-AI
system.

Relevant evaluation should therefore consider not only output accuracy, but also clinically important
patterns of human-AI interaction, including over-reliance, under-reliance, shared human-AI error, and the
ability of clinicians to recognize and correct erroneous or incomplete outputs. This is particularly important
where the clinician contributes verification, contextual interpretation, comparison with prior
examinations, and clinical synthesis beyond the AI-generated output.

15. Are there ways in which performance might be assessed relative to the care, technology, or course
of action likely to occur in the absence of the device, rather than to the comparators described in
this section? What approaches might be used to identify and justify a comparator such as unaided
clinical judgment, delayed specialist review, or no intervention?

Mosaic believes comparison with the care likely to occur in the absence of the device can provide useful
context, particularly when the intended benefit is to improve access, reduce delays, or augment existing
care. However, the comparator should not simply reflect the limitations of the current care pathway if
those limitations themselves represent a remediable systems problem.

A device should not be considered adequately validated solely because it improves upon a lower-quality
or resource-constrained baseline when a higher-quality clinical pathway is reasonably achievable. For
example, if emergency physicians are making decisions without timely radiology interpretation, AI
augmentation may improve performance relative to unaided emergency physician assessment. However,
that does not establish equivalence to, or superiority over, care that includes timely radiologist
interpretation, particularly when the radiologist is also trained to use the AI effectively.

CDRH should therefore distinguish between a descriptive counterfactual, meaning what currently happens
in the absence of the device, and a clinically appropriate comparator, meaning the care pathway that
should reasonably be available. Counterfactual comparisons may help demonstrate incremental value, but
they should not be used to normalize avoidable deficiencies in access, workflow, or turnaround time.

For assistive AI, evaluation should therefore consider both the existing baseline and an appropriate clinical
standard. Where the device is intended to compensate for delays in specialty care, sponsors should justify
why AI augmentation is being evaluated as an alternative to improving access to the relevant specialist
rather than merely managing around the underlying systems problem. As healthcare moves toward care
models designed to reduce costs, those savings must not come at the expense of care quality. AI should
support more efficient delivery of clinically appropriate care, rather than institutionalize a lower-quality
pathway simply because it costs less or improves upon an inadequate baseline.

16. Should independent third parties be involved in some or all aspects of a competency-based
approach, including device benchmarking and clinical confirmation? If so, in what ways might
qualified, independent third-party participation contribute to this approach, and what qualifications
and independence criteria should apply? Are there aspects of the assessment for which third-party
involvement would be impractical or inadvisable? What safeguards or program design features
would be critical to prevent third-party participation from limiting competition or preventing
innovation?

Mosaic supports the use of qualified independent third parties where they improve scalability, objectivity,
or access to clinical and technical expertise, but third-party involvement should not become a mandatory
additional layer for every aspect of competency-based evaluation.

Third parties can be particularly valuable for benchmark development, common performance metrics,
independent clinical adjudication, data standards, monitoring infrastructure, and other functions that
benefit from shared methods or independence. Qualified external organizations can help healthcare
organizations that lack the infrastructure or expertise to perform certain validation, monitoring, or
adjudication activities internally. Such support can extend access to rigorous evaluation and governance
capabilities without requiring every healthcare organization to independently develop the same
infrastructure and expertise.

CDRH could establish minimum expectations for the qualifications, independence, transparency, and
methodological rigor of third parties participating in device benchmarking or clinical confirmation. Such
expectations should promote consistency and credibility in regulatory evidence generation without
transferring responsibility for local clinical governance to FDA or to the third party.

Manufacturers should remain accountable for product-level evidence, performance, changes, and
applicable safety obligations. Manufacturers may also provide technical infrastructure, data access,
monitoring tools, and product-specific information necessary to support evaluation. However, when
clinical judgment is required to establish reference standards, adjudicate device outputs, or determine
clinical acceptability, the assessment should incorporate appropriately qualified clinical reviewers who are
independent of the manufacturer.

Healthcare organizations should remain accountable for determining whether and under what conditions
an authorized device is deployed within their clinical environment. Clinical evaluation, interpretation of
local findings, and governance activities may be performed internally or supported by qualified
independent clinical governance partners, but the healthcare organization should retain access to the
relevant evidence and ultimate authority to expand, restrict, suspend, or discontinue local use. Academic
medical centers that have both clinical and technical capabilities in the evaluation and implementation of
GenAI devices could also be considered as “embedded evaluators”, as has been proposed in modern
discourse by frontier labs.
Third-party participation should therefore supplement rather than diffuse existing accountability. CDRH
should avoid creating a framework that depends on a single designated evaluator, proprietary
methodology, or mandatory third-party intermediary because such requirements could create
unnecessary costs, limit competition, and slow innovation. A flexible framework should recognize multiple
qualified entities and allow healthcare organizations with appropriate expertise and infrastructure to
perform relevant activities internally. As an alternative to independent third-party evaluators, the FDA
could explore partnering with the groups described above in an internal capacity to ensure the rigor of
evaluation and allow flexibility in evaluating diverse intended uses.

17. CDRH is exploring whether a competency-based approach could be applied to devices that
incorporate a variety of underlying model architectures, such as multimodal vision-language models
and generative or predictive world models. Are there aspects of the described approach that might
be ineffective or inapplicable to these devices? What additional or distinct considerations might
need to be addressed?

Mosaic supports applying a common, risk-proportionate competency-based framework across different
underlying model architectures. Regulatory expectations should be driven primarily by the intended
clinical function, degree and independence of device activity, and consequences of an incorrect output
rather than by model architecture alone. The final user-facing device, as configured for clinical use, should
remain the unit of evaluation.

Different architectures may, however, introduce additional failure modes that should be addressed within
the same framework when they are relevant to the intended use. For multimodal systems, evaluation
should assess whether information from different modalities is appropriately integrated, whether
conflicting or missing inputs are recognized, and whether the device provides sufficient traceability to
support clinical review both at the point of care and for quality oversight. For systems that rely on external
foundation models or other external components, evaluation should also address how changes to those
components could affect clinical performance and how such changes will be detected, assessed, and
managed.

More autonomous or agentic systems may require additional evaluation because risk can arise from a
sequence of outputs and actions rather than from a single response. Relevant considerations may include
task-chain validity, external tool permissions, human authorization for clinically significant actions,
stopping rules, step-level auditability, and mechanisms for rapid disablement or rollback. Such additional
requirements should remain proportionate to the degree of autonomy and potential consequence of error
rather than being applied to all devices using the same underlying architecture.

CDRH should therefore avoid creating architecture-specific regulatory frameworks when the existing
competency-based and two-axis risk approaches can accommodate the relevant risks. Model architecture
and intended use should inform what competencies, failure modes, and safeguards need to be evaluated
but should not independently determine the regulatory burden.

Section VI: Postmarket Monitoring for GenAI-Enabled Devices

18. CDRH is considering whether it may be appropriate to accept greater premarket uncertainty
regarding a GenAI-enabled device’s benefit-risk profile through greater reliance on postmarket
monitoring. Under what conditions might such an approach be appropriate, and what
 characteristics of a monitoring program would need to be in place to justify reduced premarket
evidence? Are there device types or risk profiles for which this approach would not be appropriate?

Mosaic supports greater reliance on postmarket evidence generation when a device has met an
appropriate, risk-proportionate threshold for initial authorization and when residual uncertainties can be
safely evaluated after deployment. Greater premarket uncertainty should not be viewed as lowering
evidentiary expectations; rather, it shifts appropriate portions of evidence generation from premarket
evaluation to structured real-world assessment following deployment.

Such an approach is particularly appropriate for lower-risk, assistive GenAI-enabled devices in which a
qualified clinician remains able to independently evaluate the underlying clinical information and
intervene before an incorrect AI output results in significant harm. It may be particularly valuable for
devices whose real-world performance depends on local patient populations, workflow integration,
clinician interaction, or other factors that cannot be fully reproduced during premarket evaluation.

Greater reliance on postmarket evidence should generally be coupled with controlled or limited initial
deployment rather than immediate unrestricted use. Monitoring should evaluate both device
performance and the broader human-AI clinical system and should include scalable, automated
surveillance, predefined safety and performance signals, targeted clinician review and adjudication, clear
escalation thresholds, appropriate intervention or remediation, and reassessment after concerns are
identified. Criteria for expanding, restricting, or suspending deployment should be defined in advance. The
intensity of monitoring should remain proportionate to the device’s risk.

The approach is less appropriate for autonomous or higher-risk functions when an incorrect output could
result in significant harm before it can be detected and corrected. In those settings, greater uncertainty
should be resolved before authorization because postmarket monitoring cannot substitute for evidence
needed to establish an acceptable level of safety before patient exposure.

Monitoring programs should have clearly defined responsibility and decision authority for reviewing
signals, determining their clinical significance, implementing remediation, and reassessing performance.
Postmarket monitoring cannot justify greater premarket uncertainty if identified safety concerns cannot
be acted upon promptly.

Practical barriers to post-market evidence generation must be addressed before such evidence can justify
greater premarket uncertainty. Healthcare organizations’ privacy, cybersecurity, and contractual
requirements may limit sponsors’ access to the data needed for monitoring. These constraints should not
simply be accepted as unavoidable. Mosaic recommends that CDRH requires sponsors, within its
applicable authority, to demonstrate a feasible and durable mechanism for obtaining and evaluating the
necessary postmarket evidence before relying on that evidence to support authorization.

Such mechanisms could include defined contractual provisions for monitoring data, secure analysis within
healthcare organizations, or independent monitoring arrangements, with safeguards that distinguish
safety surveillance from secondary uses such as model training. Where existing authority is insufficient,
coordinated federal action or additional statutory authority should be considered to ensure that data
necessary for required safety and performance monitoring can be made available to appropriate parties
under defined privacy and security protections. If adequate postmarket evidence cannot be assured,
greater premarket evidence should be required. Monitoring obligations, timelines, and consequences for
failure to provide the required evidence should be established in advance.

19. Please comment on the potential approaches to postmarket performance evaluation, including
periodic re-benchmarking, sample-based clinician review, and performance degradation
monitoring. What additional approaches should CDRH consider, and how should the cadence and
triggering events for reassessment be determined?

Postmarket performance evaluation should extend beyond device performance alone and assess both
device performance and the broader human-AI clinical system. Clinically meaningful changes in safety or
effectiveness may arise from changes in model performance, patient populations, workflow, or clinician
interaction even when other components of the system remain stable.
• Periodic re-benchmarking: Re-benchmarking can provide a useful, standardized assessment of
whether device capabilities have changed over time, particularly following significant model,
foundation-model, configuration, or input-distribution changes. However, re-benchmarking
should complement rather than substitute monitoring performance in clinical practice because
benchmark performance may not detect changes arising from local populations, workflow, or
human-AI interaction. Re-benchmarking should be hypothesis-driven when a change is expected
or observed, rather than being driven by empirical temporal constraints.
• Sample-based clinician review: Clinician review should be risk-based and informed by safety and
performance signals rather than relying primarily on random case selection. Automated
monitoring may identify discordance patterns, unexpected changes in performance, unusual
workflow or user-interaction patterns, or other signals warranting focused review. Review should
include selected concordant as well as discordant cases because concordance does not
necessarily establish correctness and may conceal shared human-AI error. A limited risk-based
sample outside identified signals may also be useful to validate the monitoring process and detect
failure modes not captured by predefined signals.
• For autonomous applications, routine clinical workflows may no longer generate an independent
human reference against which AI outputs can be compared. 3 Monitoring approaches should
therefore incorporate alternative mechanisms for independent clinical evaluation, such as riskbased expert review, sampling of autonomous outputs, and longitudinal outcome assessment
when available. The reduction of routine human involvement should not result in a corresponding
loss of clinical observability.
• Performance degradation: Monitoring should address two pathways of degradation. The first is
deterioration in device performance, including changes in accuracy, calibration, reliability,
subgroup performance, or other clinically relevant failure patterns. The second is deterioration in
human-AI system performance, including reduced verification, passive acceptance, over-reliance,
inappropriate override, or under-reliance. Human-AI performance should therefore be reassessed
when meaningful changes in user behavior or workflow occur, even when model performance
remains stable.

In addition to these approaches, CDRH should encourage continuous signal-based monitoring that uses
automated surveillance to identify cases, users, workflows, or performance patterns requiring clinical
review. Relevant signals may include over-reliance or under-reliance, shared human-AI error, unsafe

3 Moy L, Vargas A, Gichoya JW, et al. Call to action: Accelerating AI-driven transformation in medical imaging and the broader health care system.

Radiology. 2026;320(2):e253974. doi:10.1148/radiol.253974.
acceptance of outputs, acceptance of hallucinated content, override patterns, subgroup or site-specific
performance changes, and evidence that prior remediation has or has not been effective. Clinical review
should determine the significance and root cause of identified signals rather than treating the signal itself
as evidence of unsafe performance.

Postmarket monitoring should function as a continuous learning and governance process in which signal
detection, clinical review, intervention, and reassessment are connected rather than performed as isolated
activities. Monitoring programs should have predefined pathways for escalation and remediation with
authority to modify education, workflow, safeguards, configuration, or conditions of use when clinically
meaningful concerns are identified. Monitoring without a mechanism for intervention and reassessment
is not governance.

The cadence of reassessment should be risk-proportionate rather than uniform. Periodic review may be
appropriate to confirm continued performance, but reassessment should also be triggered by meaningful
events such as significant model or configuration changes, changes to an underlying foundation model,
performance degradation, emerging failure modes, changes in error patterns or subgroup performance,
material workflow or population changes, or evidence of unsafe human-AI interaction. The frequency and
depth of reassessment should reflect the device’s risk, degree of autonomy, deployment scale, rate of
change, and evidence generated through ongoing monitoring.

20. Please comment on whether the proposed approaches to postmarket monitoring can be facilitated
by machine-based supervisory agents. What considerations, including the evaluation and reliability
of the supervisory agent itself, should CDRH take into account for such an approach?

Mosaic supports the use of machine-based supervisory agents to facilitate postmarket monitoring,
particularly for continuous surveillance and signal detection. Given the scale of AI deployment,
comprehensive manual review of all cases is neither feasible nor necessary. Effective oversight should
combine automated monitoring with targeted clinician review, preserving scalability while maintaining
appropriate clinical judgment and accountability.

Machine-based supervisory agents may be particularly useful for identifying unusual performance
patterns, workflow anomalies, changes in human-AI interaction, discordance trends, subgroup or sitespecific variation, and other safety or performance signals that warrant further review. Automated
surveillance can then prioritize cases, users, workflows, or deployment environments for targeted clinical
evaluation rather than attempting comprehensive manual review of all device activity.

The supervisory agent itself should be evaluated and monitored as part of the oversight system but
independently of the actual device. The MDDT pathway may be a prudent choice for general-purpose
supervisory agents whose schema can be easily modified across intended use settings. CDRH should
consider whether the agent reliably detects the signals it is intended to identify, the frequency and
consequences of false-positive and false-negative alerts, performance across relevant subgroups and
deployment environments, and whether changes to the monitored device or clinical workflow affect
supervisory-agent performance. The agent’s thresholds, configuration, and version should be traceable,
and its performance should be periodically validated through targeted expert review, including review of
cases the supervisory system did and did not flag.
Machine-based supervisory agents should support, not replace, clinical judgment. Signals identified
through automated monitoring should be evaluated by qualified clinical teams to determine their clinical
significance and whether they reflect device-related, data-related, technical, workflow, human-AI
interaction, or local deployment factors. Clinical teams should determine whether investigation,
remediation, modification of monitoring thresholds, or reassessment is warranted.

Machine-based supervision therefore fits within a broader monitoring-to-action process: automated
surveillance identifies potential signals, targeted clinical review determines their significance, and
substantiated concerns lead to appropriate intervention and reassessment. The supervisory system should
itself remain subject to ongoing validation so that reliance on automation does not create an additional
unmonitored source of risk.

21. What roles might clinicians, healthcare institutions, professional societies, standards-setting
bodies, and other stakeholders appropriately play in postmarket monitoring, and how can these
roles be structured without diffusing manufacturer accountability?

Manufacturers, healthcare organizations, clinical AI and specialty teams, end users, professional societies,
standards-setting bodies, and qualified independent organizations each have distinct roles in postmarket
monitoring. These roles should be structured to preserve manufacturer accountability for the product
while recognizing that important aspects of real-world and human-AI performance can only be observed
within the clinical environments where devices are deployed.

• Manufacturer: Monitor product-level performance, changes, failures, and safety using
information available across deployments; investigate complaints, adverse events, and emerging
failure modes; communicate model updates, limitations, and safety concerns; and provide the
information, data access, and technical capabilities necessary to support independent clinical
monitoring and investigation.
• Healthcare Organization: Maintain local deployment oversight, workflow integration, monitoring
infrastructure, data access, and escalation pathways; evaluate whether the AI remains
appropriate for the local patient population, workflow, and technical environment; and retain
authority to expand, restrict, suspend, or discontinue local use.
• Clinical AI / Specialty Teams: Provide clinical oversight of deployed AI systems; evaluate device
and human-AI system performance through adjudication, contextual interpretation, and
assessment of clinically meaningful signals that cannot be fully captured through automated
monitoring alone; determine the clinical significance and likely cause of identified concerns; and
recommend education, remediation, deployment restrictions, additional validation, suspension,
or retirement when warranted.
• End Users: Use AI within its authorized workflow and intended use, independently evaluate
outputs when qualified to do so, report unexpected errors or safety concerns, and participate in
review, education, and remediation when appropriate.
• Professional Societies: Develop clinically grounded taxonomies, monitoring approaches, best
practices, and guidance that promote consistency across organizations while reflecting specialtyspecific clinical requirements.
• Standards-Setting Bodies: Develop interoperable technical and data standards that enable
consistent capture, exchange, auditability, and comparison of information needed for postmarket
monitoring across devices and healthcare environments.
 • Qualified Independent Organizations: Support healthcare organizations that lack sufficient
infrastructure or expertise by providing monitoring infrastructure, analytics, case identification,
clinical adjudication, or other governance capabilities. Such organizations should supplement
rather than replace healthcare-organization accountability and should remain sufficiently
independent of the manufacturer when performing clinical evaluation or adjudication.

Manufacturer postmarket surveillance should not replace local clinical monitoring, and local monitoring
should not diffuse manufacturer accountability for product-level safety and performance. Manufacturers
remain responsible for the device; healthcare organizations remain responsible for determining whether
and under what conditions an authorized device is used within their clinical environment.

Postmarket monitoring should evaluate both device performance and the broader human-AI clinical
system. Behavioral and workflow measures, including over-reliance, under-reliance, unusual acceptance
or override patterns, and shared human-AI error, should be treated as signals for clinical review rather
than standalone measures of clinician performance.

Effective postmarket governance should connect signal detection to clinical review, determination of
significance, intervention, and reassessment. Clear assignment of responsibility and decision authority is
essential so that distributed participation strengthens monitoring without diffusing accountability.

22. CDRH envisions that a manufacturer’s premarket competency-based assessment might serve as a
baseline against which post-deployment modifications could be re-evaluated. How might the
extent of re-benchmarking or other evidence be scaled to the nature and expected impact of a given
modification? For example, are there categories of change that might not significantly affect safety
or effectiveness of a GenAI-enabled device, or might be appropriately managed within a sponsor’s
quality management system versus requiring FDA premarket review and authorization before
implementation? Are there categories of changes that might be appropriate for inclusion in a PCCP?

Mosaic supports a risk-proportionate approach in which the extent of re-benchmarking or other evidence
is determined by the nature of the modification, the competencies potentially affected, and the reasonably
expected impact on safety and effectiveness. A modification should not automatically trigger repetition of
the full premarket assessment. Rather, sponsors should evaluate which previously established capabilities,
failure modes, safeguards, or intended-use assumptions could reasonably be affected and focus
reassessment on those areas.

The same two-axis risk framework used for initial evaluation can help guide change assessment.
Modifications that do not alter intended use, degree of device autonomy, clinical workflow, intended user,
or clinically meaningful performance, and that are not reasonably expected to affect safety or
effectiveness, may be appropriately managed within the sponsor’s quality management system with
documented verification and ongoing monitoring.

Changes that could materially affect clinical performance, introduce new failure modes, alter the intended
user or workflow, change the degree of device activity or autonomy, or modify clinically important
safeguards should require more extensive evidence. Such evidence may include targeted re-benchmarking
of affected competencies, clinical evaluation where necessary, and enhanced post-deployment
monitoring. Changes that materially alter intended use or substantially increase the potential
consequence of an incorrect output may warrant FDA review before implementation.
Predetermined change control plans (PCCPs) may be particularly useful for categories of anticipated
modifications that can be described in advance, for which the expected effects on device behavior are
reasonably bounded, and for which predefined evaluation methods, acceptance criteria, and monitoring
requirements can be established. Examples may include defined retraining or model-update processes,
calibration changes, or other anticipated modifications for which the manufacturer can prospectively
specify how safety and effectiveness will be assessed.

GenAI-enabled devices that rely on third-party foundation models require additional attention because
clinically meaningful changes may occur outside the device manufacturer’s direct control. Manufacturers
should have mechanisms to detect relevant changes in underlying dependencies, determine whether
those changes affect previously established competencies or safeguards, and trigger proportionate
reassessment at a time appropriate to the expected risk of the change, including before broader use when
the change could materially affect safety or effectiveness.

CDRH should avoid requiring complete re-evaluation for every modification. The goal should be to reassess
the competencies, safeguards, and clinical performance characteristics reasonably affected by the change,
with the intensity of review proportionate to the likelihood and consequence of altered clinical
performance.

23. How might PCCP concepts or other change-control approaches be adapted for GenAI-enabled
devices when the nature or scope of future modifications cannot be fully prespecified?

PCCPs can provide a useful mechanism for managing anticipated modifications to GenAI-enabled devices,
but the exact nature of future changes may not always be entirely foreseeable. In those circumstances,
CDRH should allow PCCPs to define the permissible scope of change, the competencies or safeguards that
must remain intact, the methods and acceptance criteria for evaluating modifications, and the thresholds
that would trigger additional clinical evaluation or FDA review.

PCCPs should be complemented by postmarket monitoring, clinical review, and reassessment because
some modification effects may only become apparent in real-world use. Change-control approaches
should therefore focus not only on whether a modification was anticipated but also on whether it alters
clinically meaningful performance, introduces new failure modes, changes intended use or user
interaction, or materially affects the device’s risk profile.

When a modification falls outside the predefined boundaries of the PCCP or produces an unexpected
change in safety or effectiveness, additional evaluation and, where appropriate, FDA review should be
required.

24. For devices built on third-party foundation models, changes to the underlying model may be
initiated by the third-party model developer rather than the device manufacturer. How can a
manufacturer detect, evaluate, and respond to such changes in a timely manner? What
mechanisms—for example, contractual, technical, or through a PCCP—could provide reasonable
assurance that third-party developer-initiated changes do not compromise the safety or
effectiveness of the device?

GenAI-enabled devices that depend on third-party foundation models create unique change-control
challenges because clinically meaningful performance changes may originate outside the direct control of
the device manufacturer. This dynamic reinforces the need for lifecycle monitoring, dependency
management, and timely reassessment. Fundamentally, however, it is a responsibility of the sponsor to
ensure that a device continues to be safe, deterministic, and reproducible, independent of the individual
components used within the device.

The pace of updates for modern frontier models far outpaces the pace of updates for FDA-authorized
medical devices. Consequently, there is a real risk that third-party foundation models that may be
underpinning a given device are deprecated shortly after a device is cleared and being deployed broadly.
In this setting, an improved foundation model should ostensibly be better, but there is extensive testing
required to ensure feature compatibility even for foundation model upgrades.

Manufacturers should maintain mechanisms to detect and characterize changes to underlying foundation
models, including version tracking, technical change detection where feasible, and contractual
requirements for timely notification of updates that could affect device behavior, safety, or effectiveness.
Reliance on postmarket performance signals alone may be insufficient if clinically important changes can
be identified earlier through direct notification or version control.

When a third-party foundation model changes, the manufacturer should assess whether the modification
could affect previously established device competencies, safeguards, failure modes, intended use, or
clinical performance. The extent of reassessment should be proportionate to the expected impact of the
change and may include targeted re-benchmarking, clinical evaluation, enhanced post-deployment
monitoring, or temporary restriction of use when appropriate. The manufacturer should notify the FDA
about these changes and their impact on potential device outputs. Given that most devices should have
high-quality continuous integration and continuous delivery (CI/CD) pipelines for automatic bug
assessment, these should be leveraged to identify both small and large deviations in model performance.

PCCPs may provide a useful mechanism for anticipated categories of third-party model changes when the
permissible scope of change, evaluation methods, acceptance criteria, and escalation thresholds can be
defined in advance. Changes outside those predefined boundaries, or changes associated with unexpected
degradation in safety or effectiveness, should trigger additional evaluation and, where appropriate, FDA
review.

Manufacturers should also have clearly defined processes for communicating clinically meaningful
changes to healthcare organizations and supporting investigation and remediation when third-party
updates affect deployed devices.

Section VII: Other Topics

25. Would voluntary Foundation Model MAFs be practical for foundation model developers to provide
and to keep current, and what content might they include? Given that participation would be
voluntary and that model developers may have limited incentive to disclose safety-relevant
information, what would make such a program sufficiently useful for premarket review? Are there
alternative mechanisms CDRH should consider for obtaining information about underlying
foundation models?

Mosaic does not believe that a Foundation Model MAF will be useful or practical for the scenario described
by the FDA. Our opinion is shaped by the fact that knowledge of the underlying foundation model is
insufficient to identify how the foundation model is used thereafter by a manufacturer. This limitation
arises because multimodal foundation models can have substantially different performance as a function
of the prompt, the decoding approach, temperature, the reasoning budget, and the extent of memory
provided. Simply knowing system specifications for an MAF will not be beneficial for understanding how
the model will react in a downstream medical device use case.

Since the developer of the foundation model and the manufacturer will be independent parties with no
knowledge or interaction with one another, it will be challenging to evaluate how the two tools interact.
Consequently, it is unclear whether there can be meaningful information obtained by the FDA or the
manufacturer around real-world premarket or postmarket performance. This uncertainty is further
compounded by the discussion paper’s concern about foundation model developers’ limited incentives to
disclose safety-relevant information, including information about alignment and red-teaming efforts.

Acknowledging that this evaluation challenge is difficult to resolve, a potential solution could include third
parties that perform independent verification and validation (IV&V) of devices, akin to programs currently
being used by DARPA and ARPA-H. IV&V could be conducted by existing neutral entities (NIST, national
laboratories, etc.), or by nonprofit or for-profit organizations in this IV&V environment. This environment
would not be accessible to the device manufacturer and would only be accessible by the FDA and the IV&V
team. It would consist of a sandbox with real clinical data related to the intended use to assess the claims,
performance, and shortcomings of submitted medical devices. Such an approach provides the FDA with a
first-person ability to interact with the device, rather than being limited to the current paradigm of data
provided by the manufacturer. Mosaic believes that such a future approach could improve auditability and
assessment of the generalizability of devices.

26. Are there additional considerations that inform the premarket and postmarket evaluation of
agentic GenAI-enabled devices, beyond those applicable to nonagentic GenAI-enabled devices?
How should the elevated risk associated with autonomous multi-step action, tool use, and reduced
opportunity for human review be reflected in acceptance criteria and oversight?

Mosaic believes the same risk-proportionate framework should apply to agentic and nonagentic GenAIenabled devices, but agentic systems may require additional evaluation because risk can arise from a
sequence of actions rather than from a single output. The degree of autonomy, the clinical significance of
actions taken, and the opportunity for meaningful human review should directly influence acceptance
criteria and oversight intensity.

Premarket evaluation should assess not only the correctness of individual outputs but also the validity and
safety of the overall task chain. Relevant considerations may include whether the agent remains within its
intended scope, how it selects and uses external tools, whether clinically significant actions require human
authorization, whether appropriate stopping rules are in place, and whether each step in the sequence is
sufficiently auditable to reconstruct what occurred.

Acceptance criteria should become more stringent as autonomy increases and opportunities for human
intervention decrease. For higher-risk actions, CDRH should consider requiring stronger evidence of taskchain reliability, predefined boundaries on permitted actions, effective escalation or deferral behavior, and
mechanisms for immediate interruption, disablement, or rollback when unsafe behavior is detected.
Postmarket monitoring should evaluate both individual actions and patterns across multi-step workflows.
Monitoring should be capable of identifying unexpected action sequences, repeated failures,
inappropriate tool use, boundary violations, and changes in agent behavior over time. Clinically
meaningful signals should trigger review, intervention, and reassessment with monitoring intensity
proportionate to the risk and autonomy of the system.

Where agentic systems are used in settings in which a qualified clinician can review and authorize clinically
significant actions before they occur, some risk may be mitigated through human oversight. Where such
review is absent or ineffective, more uncertainty should be resolved before deployment because
postmarket monitoring cannot substitute for safeguards needed to prevent irreversible or highconsequence harm.

Conclusion

Mosaic appreciates CDRH’s efforts to develop a flexible, risk-proportionate framework for GenAI-enabled
medical devices. We support an approach that establishes appropriate premarket evidence while
recognizing that safety and effectiveness must also be evaluated throughout deployment and real-world
use. Regulatory expectations should remain proportionate to device activity and clinical risk, preserve
meaningful clinical confirmation, and support scalable postmarket monitoring of both device and humanAI system performance.

As GenAI-enabled medical devices continue to evolve, maintaining clear accountability, clinically
meaningful oversight, and mechanisms that translate monitoring into action will be essential to enabling
innovation while protecting patients.

Respectfully Submitted,

Dr. Nina Kottler, MD, MS, FSIIM, FAIM, Chief Medical AI Officer, Mosaic Clinical Technologies
Dr. Anand Singh, MD, FACR, Associate Director, Mosaic Clinical Technologies
Dr. Akshay Chaudhari, PhD, Co-Founder Cognita Imaging (acquired by Mosaic Clinical Technologies) and
Associate Professor of Radiology and Biomedical Data Science at Stanford University
Justine Jamero, MHA, Associate Director, Health Policy & Advocacy