Tempus AI (Elaine Katrivanos)

See attached file

Tempus AI
600 W. Chicago Ave
Chicago, IL 60654
Re: Docket No. FDA-2026-N-7874, Considerations for the Regulation of Generative
AI-Enabled Medical Devices: Discussion Paper and Request for Feedback
October 19, 2026
Dockets Management Staff
Food and Drug Administration
5630 Fishers Lane, Rm. 1061
Rockville, MD 20852

To Whom It May Concern:
Tempus is a technology company advancing precision medicine through the practical
application of artificial intelligence in healthcare. With one of the world’s largest libraries
of multimodal data, and an operating system to make that data accessible and useful,
Tempus provides AI-enabled precision medicine solutions to physicians to deliver
personalized patient care and in parallel facilitates discovery, development and delivery
of optimal therapeutics.
Tempus appreciates the opportunity to comment on CDRH's discussion paper on
generative AI-enabled medical devices. We believe that the competency-based approach,
with device benchmarking followed by clinical confirmation, and the two-axis risk
framework, would benefit from clarification on how the consequence-of-error axis
applies to general-purpose functions. Existing risk management frameworks, such as ISO
14971, should remain the foundation, with new GenAI-specific tools added only where
needed to give clearer guidance on baseline testing expectations. FDA should make the
framework predictable and proportionate: define discrete risk categories, and publish a
default mapping from each category to the benchmarking elements and level of clinical
confirmation expected, with flexibility for sponsors to justify use-case-specific selections,
so that lower-risk functions are not necessarily held to the full set of evaluations and
reviewers apply the same expectations for devices with the same risks. We also ask FDA
to recognize design features that measurably lower risk, including outputs grounded in
the patient's data and cited sources, traceable reasoning, constrained outputs that are
reproducible within a validated tolerance, and guardrails that defer or escalate when a
request falls outside the device's scope.
The benefit-risk assessment should be judged against the care a patient would actually
receive without the device, which in many settings is delayed or unavailable specialist
input, a generalist working under real-world time constraints, or unguided online searches
and general-purpose AI tools used by lay users, rather than an idealized expert.
Benefit-risk assessments should therefore account for the tools patients already use.
Patient-facing functions should be assessed on the associated risks and safeguards they
include rather than presumed higher risk. We believe that premarket evidence should be
balanced with postmarket monitoring where errors can be detected, and PCCPs should be
authorized for classes of change, including replacement of a third-party foundation model
with robust re-testing, bounded by the intended use and validated performance. Finally,
we encourage FDA to build on existing AI/ML frameworks rather than add new
mandatory layers such as third-party certification, and to work with other agencies on
voluntary, incentive-based data sharing that would give manufacturers additional
real-world performance signals to complement existing complaint-handling processes.
Our responses to the individual discussion questions are provided in Attachment 1.
Thank you for the opportunity to comment. Tempus would welcome continued
engagement with FDA as it develops its approach to generative AI-enabled medical
devices. Please do not hesitate to contact Elaine Katrivanos at
elaine.katrivanos@tempus.com.
Sincerely,
Elaine Katrivanos
Vice President, Regulatory Affairs
Tempus AI
Attachment 1: Tempus AI Responses to FDA Discussion Questions
Tempus has responded to a subset of the questions most relevant to our experience.
FDA’s questions are reproduced in bold, each followed by Tempus’s response.

Risk framework (Section IV, Q1-6)
Q1 — Does the two-axis risk framework, organized around device activity and the
consequence of relying on an incorrect output, appropriately capture the dimensions
most relevant to the risk of a GenAI-enabled software function? If there are
additional dimensions—such as the reversibility of a resulting action, the availability
of downstream safeguards, the time pressure of the deployment setting, or the
traceability of the output (i.e., to primary source materials)—that should be
represented in a risk framework, please describe and provide examples of how they
should be represented. ​
As other commenters have stated, existing frameworks such as ISO 14971 and the
IMDRF software risk guidance should remain the foundation for assessing GenAI device
risk. A GenAI-specific framework is most useful where it gives clearer guidance on
baseline testing expectations, with flexibility for sponsors to justify a different approach.
Activity and consequence of error are useful dimensions for assessing risk. However,
consequence-of-error can be difficult to apply to certain more general functions, because
the same function could produce errors ranging from minor to life-threatening. FDA
should explain how such functions would be categorized and give examples of functions
that would qualify as lower risk. In addition, two output characteristics change risk
enough to belong in the framework as modifiers: 1) Whether the output is grounded in
source truth, and traceable to the patient's data and a citable clinical source versus a
“black box” algorithm that cannot be independently verified; and 2) Whether the device
runs in a controlled, low-variability configuration with outputs that are reproducible
within a pre-specified, validated tolerance (for example, outputs are only selected from a
fixed set of options), similar to traditional AI/ML devices. Not every GenAI device can
operate this way, but devices designed to do so should be recognized as lower risk than
open-ended configurations. Both are discussed further in response to Q2. We also believe
that the reversibility of the resulting action belongs in the framework. An error that a user
can catch and correct before it affects care, such as a draft that a clinician reviews before
signing, carries less risk than one that autonomously triggers an action that cannot be
undone. Reversibility also bears on the rollback and human-confirmation controls
discussed in response to Q18 and Q26.
Q2 — CDRH seeks input on how to account for the spectrum of GenAI-enabled
informational functions that vary in the degree to which they direct a user to a
particular action (i.e., between “non-directive” and “action-directing”). What
characteristics of an output—such as its wording, specificity, personalization, or
context—could be considered as modifiers of the risk associated with an
informational function, after accounting for the device’s overall functionality and
intended use? What additional information would provide manufacturers with
sufficient clarity and predictability around risk assessment for informational
functions while recognizing that directiveness may exist along a continuum rather
than as a binary distinction?
FDA should recognize two modifiers that meaningfully lower risk when present:
●​ Grounding in source truth and output traceability: There is a significant
difference when comparing a “black box” algorithm that cannot be independently
verified to an output that can be checked. A response that is generated from, and cites,
the patient's own data and an identifiable clinical source is materially lower-risk than
an output that is not grounded in the patient's data and does not include a reasoning
trace, because it lets the user verify the basis rather than rely on the assertion alone.
The same reasoning applies to devices that draw on many real-world data inputs, such
as a full electronic health record (EHR). The distinction should not turn on the
number of inputs, but on whether the output is one the user could reach on their own
given enough time to review the available information (e.g., a detailed EHR review
against clinical guidelines), in which case the device works as an efficiency tool.
Confabulation/hallucination risk can be reduced when every claim traces to a source.
Grounding and traceability are related but distinct; grounding ties the content of an
output to the patient's cited data and identified clinical sources, while traceability
preserves the reasoning trace so that a clinician (or a separate verification model), can
check how the output was derived. Grounding does not require the full reasoning
trace, and each should be recognized as a separate risk-reducing feature.
●​ Output constraint and configuration control: A device with a fixed prompt, fixed
retrieval sources, defined guardrails, constrained decoding, and schema-based outputs
(e.g., selecting from a fixed set of options rather than generating free text) can
produce behavior that is reproducible within a pre-specified tolerance and that
behaves similarly to traditional medical device algorithms. Because GenAI outputs
cannot be guaranteed to be fully deterministic (even at a temperature of zero),
reproducibility should be measured empirically (e.g., the agreement rate across
repeated runs on the same inputs) against a pre-specified threshold. The threshold
need not require perfect agreement and can be set relative to the consistency of the
clinicians who perform the task today, recognizing that this will differ by clinician
type (for example, specialist versus generalist) and may not be measurable for every
use case. Devices that meet the threshold could reasonably be evaluated under
existing AI/ML frameworks rather than GenAI-specific requirements. Such devices
should be recognized as having a different risk profile compared to an open-ended
configuration that can vary from run to run, and the framework should treat
configuration control as a risk-reducing design choice.
While it is true that the FDA-defined criteria may exist along a continuum rather than as a
binary distinction, this is not a novel concept for medical device development. To avoid
providing insufficient direction and rules that may be applied differently across FDA
review teams, it is important to define discrete categories that set baseline testing
expectations. A supporting rationale could be provided by the manufacturer and reviewed
by FDA to confirm whether the correct subset of tests was chosen based on the risk and
associated controls in place. Clear, predictable risk definitions and premarket/postmarket
requirements established ahead of time will provide companies with the appropriate
expectations needed to invest in the development of novel Gen AI medical devices.
Conversely, ambiguity or inconsistent interpretation may create regulatory
unpredictability that could delay patient access to beneficial technologies. Predictable
criteria also serve patients and safety oversight; when devices with the same risks are
placed in the same category, FDA can apply consistent controls, and patients gain timely
access to the devices that meet them.
Q3 — CDRH seeks input on whether and when a GenAI-enabled function that
results in delivery of clinical information to patients, as opposed to HCPs, could
present different or higher risks, while also recognizing the potential benefits
associated with improved patient empowerment, engagement, and access to clinical
information. What device characteristics, output features, or safeguards might
mitigate risks that could arise when a user lacks the domain knowledge to
independently evaluate an output, without unnecessarily underestimating patient
capability? ​
Patient-facing delivery of medical information is already common among existing
non-device Gen AI software, and it is important to consider patient empowerment and
access. The risk that the user cannot independently judge the output exists, but is similar
to existing HCP-facing devices that provide a “black box” algorithm output rather than
being specific only to patients. The safeguards that most directly address the concern are
the same ones discussed in Q2 (e.g., grounding in source truth and traceability so claims
are checkable), plus explicit scope limits, health-literacy features (e.g., plain-language
explanations and definitions of medical terms at an accessible reading level), deferral
when the input is outside scope or the model's confidence is low, and clear escalation to a
clinician for anything that could be time-sensitive. Risk should be assessed on whether
these safeguards are present and validated with risks mitigated to acceptable levels, and
not strictly on patient-facing status. It is important not to underestimate patient capability;
many patients are already managing complex conditions with poorly structured
information, and access to patient-facing Gen AI medical devices may offer significant
benefits over the status quo available today. General-purpose GenAI tools are already
widely used for health information, so benefit-risk assessments should consider the
sources patients would otherwise rely on. A validated device with the safeguards
described above may lower risk compared with what patients use today. For example, an
independent evaluation of a widely used consumer health AI tool found that it
recommended less urgent care than clinicians had assigned in a clinically meaningful
proportion of emergency scenarios (Ramaswamy et al., “ChatGPT Health performance in
a structured test of triage recommendations,” Nature Medicine 32:1671–1675, 2026;
doi:10.1038/s41591-026-04297-7). Where patient capability is a concern, it should be
addressed directly by defining the intended user population in the indications for use and
confirming it through usability testing with representative patient users when necessary.
Q4 — CDRH seeks input on whether and how the distinction between generalist
and specialist physicians might inform the assessment of risk for HCP-facing
GenAI-enabled functions. Under what circumstances, if any, might risk be affected
when an HCP who lacks the relevant clinical specialist knowledge receives
information that falls within a specialist area of practice? What device
characteristics or safeguards might mitigate such risks? ​
In addition to documenting whether the HCP user requires certain minimum
training/education to understand device output and to implement risk mitigation
measures, it's important to consider whether the generalist HCPs are involved in such
tasks as part of the current standard of care. Output features such as citations to relevant
clinical guidelines or literature can further mitigate the risk of a generalist missing
specialist knowledge, because they allow the generalist to check the basis of an output.
And in some situations, the generalists involved in patient care with less specialized
training may benefit more from the Gen AI device than the specialist, resulting in an
improved benefit-risk profile. This is especially true for devices used to screen for
diseases that may be otherwise missed for which there is low risk downstream
confirmatory testing or referral to a specialist, and clear statements that the device is not
meant to rule in or rule out disease on its own. When evaluating the benefit-risk profile of
the device, FDA should not consider only how use of the device compares to an idealized
highly expert and specialized clinician in the space, as access to such clinicians is often
difficult and delayed, especially in underserved communities. Instead, FDA should
consider a comparison to a realistic comparator (e.g., patients self-diagnosing/treating
based off websites/social media/non-device LLM, a generalist clinician without specialist
training in the condition, etc.) for which the benefit of a well designed and controlled Gen
AI device could lead to a significant improvement in patient outcomes for the majority of
patients. The comparator should also reflect the time constraints under which clinicians
actually practice, rather than simulated expert review that may be unrepresentative of
real-world physician constraints. A realistic comparator does not mean accepting
substandard care; the device should still meet prespecified acceptance criteria for
safety-critical behavior, such as recognizing and escalating emergencies, regardless of the
comparator.
Q5 — For multi-turn conversational GenAI-enabled devices that may migrate from
providing “non-directive” information to “action-directing” information over the
course of an exchange, how should risk be assessed across realistic conversational
trajectories? How could the intended use of such a device be characterized when its
behavior is emergent across a conversation? ​
The device should be evaluated across realistic multi-turn trajectories and require the
design to include guardrails that recognize the point at which a conversation becomes
action-directing and to respond with the corresponding controls (e.g., deferral, escalation,
or communication of a scope boundary). Supervisory agents (which may be incorporated
as either device components or third-party software) that monitor the conversation in real
time are one such guardrail which could identify and navigate action-directed information
through the appropriate system.
Q6 — For GenAI-enabled care escalation functions, CDRH is considering how the
evaluation may account for both under-escalation and over-escalation. How could
manufacturers characterize and weigh these two directions of error, given that they
may not be commensurable and that acceptable trade-offs may vary by clinical
context? ​
As with traditional device algorithms, the evaluation should consider the impact of both
false positives and false negatives. However, unlike traditional algorithms, GenAI
devices generally do not have a single operating point that can be tuned to trade
sensitivity against specificity. Changes in prompting and the use of deterministic rules
can shift behavior but do not give the same degree of control. These two errors should
have their own direction-specific acceptance criteria, with associated justifications that
consider the differing clinical consequences. In order to characterize whether the situation
has been under- or over-escalated, a predefined abstraction protocol may be used to
define how each output will be assessed. Where retrospective data has no single correct
escalation, the protocol can define the range of acceptable responses for each scenario.
Published methods illustrate how this can work: clinicians independently assigned a
guideline-anchored triage level to each scenario, recording a single correct level where
the evidence supported one, and two adjacent acceptable levels where both were
clinically reasonable, along with the lowest level considered clinically safe. Under- and
over-triage were then reported separately by acuity level, and nonclinical factors (e.g.,
barriers to care) were considered within each scenario to test whether they shifted
recommendations (Ramaswamy et al., 2026).

Premarket evaluation (Section V, Q7-17)
Q7 — Is the competency-based approach described above, i.e., device benchmarking
followed by clinical confirmation, a useful and appropriate framework for
evaluating GenAI-enabled devices? ​
The competency-based approach provides a useful foundation for evaluating
GenAI-enabled devices. The approach only delivers on "least burdensome" if element
selection is genuinely scaled to risk. As other commenters have stated, the presence of
GenAI alone should not trigger every element of a competency assessment, and sponsors
should be able to use established analytical and clinical validation methods where they
are appropriate. The Agency should provide several examples of a lower risk device,
including devices where risks are mitigated through design controls and through the
balance of pre-market and post-market evaluation as opposed to only comprehensive
premarket validation, to demonstrate how a subset of the ten benchmarking elements
would suffice. This will help ensure that the framework is not read as requiring the full
list of evaluations for every GenAI function regardless of risk. The more explicitly this
can be defined, the better, as this will help remove ambiguity around premarket
evaluation expectations for device developers and encourage consistency across FDA
review teams.
Q8 — How might the two-axis risk framework described in Section IV be
considered within the competency-based approach described above to help
determine the level of evidence needed for a premarket submission? ​
The risk category should be mapped explicitly to which benchmarking elements apply
and to the rigor of clinical confirmation expected to help remove ambiguity around
premarket evaluation expectations for device developers and encourage consistency
across FDA review teams. A low-risk informational function that is grounded in the
source truth, HCP-facing, and designed to defer a final decision to the HCP should draw
a smaller subset of elements and a proportionate clinical confirmation approach. FDA
should include examples of functions it would consider low risk (e.g., a review of the
medical record to identify gaps in guideline-recommended testing) to help clarify what
device types qualify under the consequence-of-error axis. Whereas an action-directing or
autonomous function should generally draw a greater number of elements relevant to its
intended use and higher-rigor clinical confirmation. For example, FDA could publish a
matrix that maps each section of the two-axis risk framework to a defined risk category
(e.g., low, medium, high) and, for each category, to the applicable benchmarking
elements and confirmation approach. The matrix should serve as guideposts that order the
benchmarking elements by risk category, with flexibility for the sponsor to justify a
use-case-specific selection, rather than as a rigid checklist. Making this mapping explicit
will help keep the approach proportionate and least-burdensome.
Q10 — CDRH recognizes that publicly available benchmarking assets may be
subject to data contamination, saturation, and limited real-world
representativeness. How should a sponsor establish that performance on a given
benchmark predicts safe and effective real-world behavior for the device’s intended
use? What evidence should support the construct validity of a benchmark used to
gate device evaluation, and what role should sponsor-developed benchmarks play
given potential concerns around independence and optimization to the test? ​
The competency-based approach proposed by FDA is not intended to demonstrate safe
and effective performance through benchmarking alone, and instead relies on a
combination of both device benchmarking and clinical confirmation. To account for these
potential limitations, use of benchmarking should be part of the process to evaluate
certain performance aspects that are unlikely to be adequately addressed through other
design and evaluation activities such as implemented risk mitigations/guardrails and
clinical confirmation. Public benchmarks can show that a device performs at least as well
as published models, while sequestered test data that is not publicly available and is
drawn from multiple independent sites can help address concerns related to
contamination and optimization to the test. Public datasets can and should be part of the
evaluation, particularly for general tasks, but should not be the sole source, both because
independence from foundation-model training is difficult to establish and because public
datasets may not have been designed for the device’s specific task.
Q11 — CDRH is considering that clinical confirmation for a GenAI-enabled device
might not require a prospective clinical study in every case, and has described above
a range of approaches of increasing rigor and patient exposure. How might a
sponsor select and justify a confirmation approach tailored to a device’s intended
use and proportionate to the device’s risk profile? Are there device types or risk
profiles for which one or more of these approaches would be insufficient or
inappropriate? How might the anticipated distribution of real-world inputs be taken
into consideration? Are there other methods of clinical confirmation that might help
inform the evaluation of GenAI-enabled devices? ​
As with traditional AI/ML devices, the approach should be selected based on the risk
profile. For lower-risk functions, retrospective evaluation on real patient inputs or shadow
deployment should generally suffice, while higher-risk functions may warrant
standardized-patient interaction, clinician adjudication, or a prospective study.
Real-world evidence, consistent with FDA's Use of Real-World Evidence guidance, may
be appropriate both for initial premarket clinical confirmation and in the postmarket
setting. Simulated inputs derived from real-world data can also supplement confirmation
for clinically important scenarios that are too infrequent to capture in any single dataset.
For multi-turn conversational devices, retrospective evaluation may be limited because
historical records cannot show how a conversation would have unfolded in response to
the device’s outputs, so evaluation of multi-turn behavior may instead rely more on
simulated-patient interactions or prospective evaluation.
Q14 — For open-ended device outputs, how should performance comparators and
acceptance criteria be selected? When a panel of qualified clinicians serves as the
comparator, how should the applicable standard (for example, the standard of care
versus the performance of a median clinician in practice) be defined and justified?
Should generalist or specialist physicians be used as a performance standard? When
should human-AI team performance, rather than the device operating alone, serve
as the basis for evaluation? ​
A median-clinician standard representative of real-world clinician responsibilities is a
reasonable comparator, but it should consider the clinical impact associated with absence
of the device and alternate ways that decisions are being made (see also the response to
Question 15). In some cases, the range of reasonable responses from competent
clinicians, rather than the median, is the more appropriate standard. An output that falls
within the bounds of what competent clinicians would do should be acceptable, in a
manner reflective of how physicians exist across this spectrum today. Published
evaluations have used this approach, treating an output as acceptable when it falls within
a clinician-defined range of acceptable responses (Ramaswamy et al., 2026).
In some situations, the generalists involved in patient care with less specialized training
may benefit more from the Gen AI device than the specialist, resulting in an improved
benefit-risk profile. This is especially true for devices used to screen for diseases that
may be otherwise missed for which there is low risk downstream confirmatory testing or
referral to a specialist, and clear statements that the device is not meant to rule in or rule
out disease on its own. When evaluating the benefit-risk profile of the device, FDA
should not consider only how use of the device compares to an idealized highly expert
and specialized clinician in the space, as access to such clinicians is often difficult and
delayed, especially in underserved communities. Instead, FDA should consider a
comparison to a realistic comparator (e.g., patients self-diagnosing/treating based off
websites/social media/non-device LLM, a generalist clinician without specialist training
in the condition, etc.) for which the benefit of a well designed and controlled Gen AI
device could lead to a significant improvement in patient outcomes for the majority of
patients. The comparator should also reflect the time constraints under which clinicians
actually practice, rather than simulated expert review that may be unrepresentative of
real-world physician constraints. A realistic comparator does not mean accepting
substandard care; the device should still meet prespecified acceptance criteria for
safety-critical behavior, such as recognizing and escalating emergencies, regardless of the
comparator.
Q15 — Are there ways in which performance might be assessed relative to the care,
technology, or course of action likely to occur in the absence of the device, rather
than to the comparators described in this section? What approaches might be used
to identify and justify a comparator such as unaided clinical judgment, delayed
specialist review, or no intervention? ​
Yes, the right comparator is often the care that would actually occur under time
constrained conditions without the device, which in many settings is delayed or
unavailable specialist input, not an idealized expert who is assumed to be present and
have perfect knowledge of the underlying patient history. Where a device extends access
to expertise that is otherwise not reachable in the relevant time window, the benefit-risk
analysis should credit that, rather than holding the device to a comparator that may not
exist in practice.
In some situations, the generalists involved in patient care with less specialized training
may benefit more from the Gen AI device than the specialist, resulting in an improved
benefit-risk profile. This is especially true for devices used to screen for diseases that
may be otherwise missed for which there is low risk downstream confirmatory testing or
referral to a specialist, and clear statements that the device is not meant to rule in or rule
out disease on its own. When evaluating the benefit-risk profile of the device, FDA
should not consider only how use of the device compares to an idealized highly expert
and specialized clinician in the space, as access to such clinicians is often difficult and
delayed, especially in underserved communities. Instead, FDA should consider a
comparison to a realistic comparator (e.g., patients self-diagnosing/treating based off
websites/social media/non-device LLM, a generalist clinician without specialist training
in the condition, etc.) for which the benefit of a well designed and controlled Gen AI
device could lead to a significant improvement in patient outcomes for the majority of
patients. The comparator should also reflect the time constraints under which clinicians
actually practice, rather than simulated expert review that may be unrepresentative of
real-world physician constraints. A realistic comparator does not mean accepting
substandard care; the device should still meet prespecified acceptance criteria for
safety-critical behavior, such as recognizing and escalating emergencies, regardless of the
comparator.
Q16 — Should independent third parties be involved in some or all aspects of a
competency-based approach, including device benchmarking and clinical
confirmation? If so, in what ways might qualified, independent third-party
participation contribute to this approach, and what qualifications and independence
criteria should apply? Are there aspects of the assessment for which third-party
involvement would be impractical or inadvisable? What safeguards or program
design features would be critical to prevent third-party participation from limiting
competition or preventing innovation? ​
Third parties have recently been started with the intent to accredit and otherwise provide
seals of approval for traditional AI/ML devices. However, even when participation is
voluntary, it could create an implied requirement if widely adopted which may
unnecessarily duplicate oversight that already exists within FDA’s regulatory
responsibilities. Use of third parties should be optional rather than mandated and caution
should be used before FDA endorsement or allowance to use a third-party approval that
may eventually gain hold as an expected minimum beyond FDA authorization. FDA
should avoid creating redundant accreditation or certification layers on top of the existing
device review; the value is in recognized standards and independent adjudication where it
is needed. Traditional AI/ML devices face the same independence concerns and have a
long history of addressing them through FDA review and without third parties; that
approach should be similarly accepted for GenAI devices. We share the concerns raised
by other commenters about the confidentiality, intellectual property and data-security
risks of requiring manufacturers to share proprietary device information with third-party
evaluators.

Postmarket monitoring and change control (Sections VI–VII, Q18-26)
Q18 — CDRH is considering whether it may be appropriate to accept greater
premarket uncertainty regarding a GenAI-enabled device’s benefit-risk profile
through greater reliance on postmarket monitoring. Under what conditions might
such an approach be appropriate, and what characteristics of a monitoring program
would need to be in place to justify reduced premarket evidence? Are there device
types or risk profiles for which this approach would not be appropriate? ​
We believe that a balance between premarket review and postmarket monitoring for Gen
AI devices is appropriate and consistent with least burdensome principles. As other
commenters have stated, postmarket evidence complements and does not replace FDA's
premarket determination of reasonable assurance of safety and effectiveness. The tradeoff
is appropriate where the device has a defined FDA-reviewed performance baseline,
pre-specified monitoring thresholds and cadence, and the ability to roll back a change or
pause deployment if performance degrades. The tradeoff also depends on the nature of
the potential harm; it fits best where errors are likely to be detected through monitoring
and their effects can be reversed or contained before they cause significant harm.
One example where additional postmarket data collection is beneficial is for subgroups.
In the premarket review, available data may support a statistical homogeneity analysis of
performance across subgroups, rather than expecting that each subgroup would be
powered like an independent study. Postmarket analyses could be prespecified to further
evaluate subgroups relevant to the intended use (e.g., age, sex, race and ethnicity, and
primary language for conversational devices), to help improve the understanding of
device performance across a diverse patient population and to confirm no true subgroup
differences exist.
A second example is premarket evidence that relies partly on simulated patient data.
Where simulated results are strong but their generalizability to the intended population is
uncertain, targeted postmarket monitoring of that functionality can help address the
remaining uncertainty.
Q20 — Please comment on whether the proposed approaches to postmarket
monitoring can be facilitated by machine-based supervisory agents. What
considerations, including the evaluation and reliability of the supervisory agent
itself, should CDRH take into account for such an approach? ​
FDA and other US government agencies could help build automated outcome feedback
that further strengthens post-market performance monitoring capabilities. Existing
complaint-handling and vigilance processes identify many performance signals today,
and outcome data could add a complementary, population-level view of performance.
Manufacturers with direct access to real-world clinical data (e.g., through data
partnerships or EHR connectivity) can monitor performance more directly, but many
manufacturers do not have that comprehensive access. Mechanisms that capture whether
an output was ultimately correct (e.g., confirmation by the clinician or by a downstream
gold-standard diagnostic result in the EHR) would add to the postmarket performance
information available from complaints, assuming the manufacturer can control for
potential confounding variables (e.g., use of the device on only sicker or healthier
populations than expected). Alongside the evaluation of individual results as complaints
or potential medical device reports (MDRs), outcome data collected across the patient
population can be analyzed in aggregate to determine whether real-world performance
remains consistent with the validated and expected sensitivity, specificity and other
performance measures, or is drifting in a way that warrants investigation. Individual
events that meet complaint or MDR reporting criteria would continue to be handled
through those processes.
Such an evaluation could be managed by a machine-based supervisory agent within the
electronic health record with data sharing to the legal manufacturer. Although certain
supervisory agents (e.g., one that monitors device performance but cannot directly change
the device’s real-time outputs or actions), may not be a medical device itself, such agents
should be thoroughly validated such that their performance can be relied upon and
sourced in an auditable ground truth. Provider participation would depend on addressing
legitimate concerns including a permissible HIPAA basis for sharing protected health
information and adherence to minimum necessary standard, the cybersecurity of any
agent operating within the EHR, and appropriate protections to providers for how
automated feedback data may be used. Voluntary participation supported by incentives
and clear data-sharing permissions, rather than a mandate, may be more likely to succeed.
Q22 — CDRH envisions that a manufacturer’s premarket competency-based
assessment might serve as a baseline against which post-deployment modifications
could be re-evaluated. How might the extent of re-benchmarking or other evidence
be scaled to the nature and expected impact of a given modification? For example,
are there categories of change that might not significantly affect safety or
effectiveness of a GenAI-enabled device, or might be appropriately managed within
a sponsor’s quality management system versus requiring FDA premarket review
and authorization before implementation? Are there categories of changes that
might be appropriate for inclusion in a PCCP? ​
As it relates to managing certain changes that are unlikely to affect safety or effectiveness
and which may be appropriately managed within a sponsor’s quality management system,
the existing mechanisms used for traditional AI/ML devices should be applied; e.g.,
changes to conversational style or tone that do not change clinical content or safety
behavior. Furthermore, the extent of benchmarking should scale with the significance of
the change. A bounded change to a single component with isolated downstream effects
should draw a targeted re-benchmark of the affected elements; whereas a
foundation-model replacement should draw a full re-benchmark because the components
function as a system.
Q23 — How might PCCP concepts or other change-control approaches be adapted
for GenAI-enabled devices when the nature or scope of future modifications cannot
be fully prespecified? ​
A GenAI PCCP should authorize classes of changes to the model, prompts, retrieval,
tools, orchestration, and guardrails. This includes a change to the foundation model when
the replacement is validated against the cleared performance baseline under the plan, and
is bounded by the device's intended use, performance requirements and a pre-specified
validation plan. It should not be required to prespecify the exact successor configuration
in advance, as it may be difficult to predict with specificity and the proposed bounding
should still be sufficient to predefine the PCCP mechanism. The change space is
combinatorial and the technology moves faster than a closed list of named modifications
can track, so the boundary should be defined by intended use and performance. This is
consistent with feedback from other commenters that a PCCP should define boundaries
and guardrails for a range of allowable changes, with the Modification Protocol and
Impact Assessment carrying more specifics to ensure continued acceptable performance.
The three PCCP elements may be adapted to accommodate this new paradigm:
●​ Description of Modifications: Identifies which components may change
(independently or together), the intended use, users, population, input/output
obligations, performance requirements, prohibited changes, and safety-critical
controls. It also could specify the change that would fall outside the plan and require a
new submission (e.g., adding autonomous medication titration, a new physiologic
input, or a new indication).
●​ Modification Protocol: Links each class of change to prespecified verification and
validation, test methods, and acceptance criteria. This may include end-to-end device
testing, safety-critical behavior, task-specific clinical correctness, subgroup and
setting performance, robustness to prompt injection and tool failure, human factors,
and reliability across repeated runs, depending on the specifics of the change. The
cleared performance serves as a baseline to which the modified version may be
compared.
●​ Impact Assessment: Addresses the interaction and cumulative effects that are
specific to GenAI. For example, simultaneous changes across layers, new failure
modes from model substitution or tool access, drift over repeated releases, and effects
on labeling, cybersecurity, privacy, and human factors.
The manufacturer will continue to maintain an auditable record of every change
implemented under the PCCP. For higher-risk devices that already submit annual reports
to FDA, such as PMA-approved devices, changes made under the plan could also be
summarized in that existing report; other lower-risk devices should not necessitate a new
reporting requirement.
Q24 — For devices built on third-party foundation models, changes to the
underlying model may be initiated by the third-party model developer rather than
the device manufacturer. How can a manufacturer detect, evaluate, and respond to
such changes in a timely manner? What mechanisms—for example, contractual,
technical, or through a PCCP—could provide reasonable assurance that third-party
developer-initiated changes do not compromise the safety or effectiveness of the
device? ​
This class of change may be appropriate in a PCCP. The manufacturer may move to a
successor model not named in the original plan, after full re-benchmarking, provided the
intended use is unchanged and performance meets the cleared baseline defined in the
Modification Protocol. Foundation model developers also make smaller changes without
changing the model version number. These generally follow standard software
engineering practice, with larger changes carrying a new release number, and can be
managed through version pinning where available and scheduled background
re-evaluation on a fixed test set to confirm that performance is maintained. Third-party
foundation model vendors should be encouraged to support medical device manufacturers
by including well documented change notification publicly and to provide a minimum
reasonable notice period before model versions are removed. The problem of model
deprecation without reasonable notice is a topic where US government agency support
may be warranted as a model developer can withdraw or retire a version on its own
timeline, which can force a GenAI medical device to be changed or removed from market
in a disruptive way. FDA should expect manufacturers to plan for forced model
migration, and should encourage model developers to provide early deprecation notice,
which may connect to the master-file concept in Q25. Device manufacturers should have
continuity plans in place showing consideration for this model access risk.
Q25 — Would voluntary Foundation Model MAFs be practical for foundation
model developers to provide and to keep current, and what content might they
include? Given that participation would be voluntary and that model developers
may have limited incentive to disclose safety-relevant information, what would
make such a program sufficiently useful for premarket review? Are there
alternative mechanisms CDRH should consider for obtaining information about
underlying foundation models? ​
Useful content includes model and system cards, a characterization of training data and
known limitations, internal guardrails, evaluation results, and, importantly, change and
deprecation notices. A MAF's main value is information a device manufacturer needs for
authorization but cannot obtain directly, such as training-data characterization and
internal safety evaluations, which FDA could review under a right of reference without
the developer disclosing it to the manufacturer. The main practical challenge beyond
questions of voluntary participation is that foundation models are updated faster than a
master file may be maintained, so a MAF should be treated as one input a sponsor
references with authorization, not as a substitute for the sponsor's own validation of the
deployed configuration.
Q26 — Are there additional considerations that inform the premarket and
postmarket evaluation of agentic GenAI-enabled devices, beyond those applicable to
non-agentic GenAI-enabled devices? How should the elevated risk associated with
autonomous multi-step action, tool use, and reduced opportunity for human review
be reflected in acceptance criteria and oversight? ​
Agentic functions inclusive of tool use and autonomous action should follow the risk
framework FDA is considering for non-agentic Gen AI devices. Tool use may be an
important mechanism to help mitigate GenAI risks. A model on its own may only emit
unverified text; a tool could be used to apply deterministic code between the model's
intent and any real-world effect, so a safety limit (e.g., a maximum recommended dose) is
enforced by the tool's code rather than requested in a prompt, and an action that violates
the limit returns an error instead of executing. The additional functionality may warrant
an evaluation addressing the permissions and boundaries on tool use, the reversibility of
actions the agent can take, and behavior when a tool fails or returns bad data. Testing
should cover the orchestration and guardrail logic, and should include adversarial cases
and tool-failure scenarios. Where an agent can take actions that cannot be undone, the
design should consider a human confirmation step before those actions, a way for the
user or deploying institution to stop the agent, and an audit log recording each action.
Rather than being assigned a higher risk category by default, multi-step planning should
be assessed using the same risk factors as other GenAI functions (which may similarly be
organized together for use in sequence), with its specific controls considered. Autonomy
is one factor in that assessment, weighed together with intended use, the consequence of
an error, reversibility, and the controls in place, rather than the deciding factor on its own.