Results by question · Docket FDA-2026-N-7874 · Comments closed October 19, 2026 · Filings posted through November 3, 2026
For every question on generative AI medical devices: what 509 filers proposed, how they answered, and where kinds of filer differed.
Answers come from the 504 filers read for content; their groups, by share
Summary reportHow this was counted
Does the two-axis risk framework, organized around device activity and the consequence of relying on an incorrect output, appropriately capture the dimensions most relevant to the risk of a GenAI-enabled software function? If there are additional dimensions—such as the reversibility of a resulting action, the availability of downstream safeguards, the time pressure of the deployment setting, or the traceability of the output (i.e., to primary source materials)—that should be represented in a risk framework, please describe and provide examples of how they should be represented.
In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes · Takeaway 8, How much an AI may do on its own should depend on the risk, filers said
Answer chart: of the 316 filers who answered (accept, with changes 181, accept, as written 9, split 1, reject 3, no verdict 122). Proposals: of the 316 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“The two-axis risk grid is sound. Four factors should adjust a function's position on the grid. They should not become new axes. …”Veritas Biomedical Analytics, Inc. The full filing →Clinicians · Calls framework too simple
“I appreciate that the FDA is offering a heuristic, but I believe the framework is too simple. It does not capture the complexity of rolling out AI in real-world settings. …”CROSS Global Research & Strategy (Shakira J. Grant) The full filing →Researchers · Adds a third factor
“We propose anticipated patterns of use as a third risk factor, alongside device activity and the consequence of relying on an incorrect or harmful output”NSF AI Research Institute on Interaction for AI Assistants (ARIA) The full filing →
“The two-axis risk grid is sound. Four factors should adjust a function's position on the grid. They should not become new axes. …”Veritas Biomedical Analytics, Inc. The full filing →Industry · Credit only measured safeguards
“Question 1. Keep the two axes, and let only measured safeguards lower risk”Anthropic PBC The full filing →Clinicians · Calls framework too simple
“I appreciate that the FDA is offering a heuristic, but I believe the framework is too simple. It does not capture the complexity of rolling out AI in real-world settings. …”CROSS Global Research & Strategy (Shakira J. Grant) The full filing →
“I write to urge FDA to add time pressure and reversibility as explicit risk modifiers in the proposed two-axis framework.”Nina Nwoko The full filing →Clinicians · Accepts, wants more factors
“The FDA's proposed two-axis risk framework is a good starting point; additional factors, however, should be addressed to more fully appraise risk.”College of American Pathologists The full filing →Industry · Wants modifiers, not axes
“The two-axis risk grid is sound. Four factors should adjust a function's position on the grid. They should not become new axes. …”Veritas Biomedical Analytics, Inc. The full filing →
“We think the two axes are the right starting point.”Intraoperative Performance Analytics Laboratory (IPAL) The full filing →Researchers · Useful but needs refining
“The two-axis framework provides a useful initial organizing tool, but FDA should examine how the additional risks identified in the discussion paper are represented within it.”Duke-Margolis Institute for Health Policy The full filing →Industry · Wants modifiers, not axes
“The two-axis risk grid is sound. Four factors should adjust a function's position on the grid. They should not become new axes. …”Veritas Biomedical Analytics, Inc. The full filing →
“The ALS Association supports a risk-based approach that considers what a GenAI-enabled medical device is intended to do and the consequences of an incorrect output.”The ALS Association The full filing →Patients · Supports the two dimensions
“The NHC supports FDA’s proposal to assess risk based on the core dimensions of device activity and the consequences of relying on an incorrect output.”National Health Council The full filing →Industry · Wants modifiers, not axes
“The two-axis risk grid is sound. Four factors should adjust a function's position on the grid. They should not become new axes. …”Veritas Biomedical Analytics, Inc. The full filing →
“We agree that risk assessment should consider how independently a device acts and the consequences of relying on an incorrect output.”Bipartisan Policy Center The full filing →Other · Useful, add context modifiers
“Question 1. The two-axis framework is useful, but its output should be modified by operational context and demonstrated control effectiveness.”VivaSecuris The full filing →Industry · Wants modifiers, not axes
“The two-axis risk grid is sound. Four factors should adjust a function's position on the grid. They should not become new axes. …”Veritas Biomedical Analytics, Inc. The full filing →
Only 19 filers in this group, so counts are shown, not shares.
Only 2 filers in this group, so counts are shown, not shares.
Of the 194 who gave a verdict, 181 accepted with changes, 9 accepted as written, 1 was split and 3 rejected.
Of the 107 who gave a verdict, 101 accepted with changes, 5 accepted as written and 1 rejected.
Of the 52 who gave a verdict, 50 accepted with changes, 1 accepted as written and 1 rejected.
All 8 who gave a verdict accepted with changes.
Both who gave a verdict accepted with changes.
Of the 25 who gave a verdict, 20 accepted with changes, 3 accepted as written, 1 was split and 1 rejected.
FDA a factor the FDA itself suggested
Only 19 filers in this group, so counts are shown, not shares.
Only 2 filers in this group, so counts are shown, not shares.
CDRH seeks input on how to account for the spectrum of GenAI-enabled informational functions that vary in the degree to which they direct a user to a particular action (i.e., between “non-directive” and “action-directing”). What characteristics of an output—such as its wording, specificity, personalization, or context—could be considered as modifiers of the risk associated with an informational function, after accounting for the device’s overall functionality and intended use? What additional information would provide manufacturers with sufficient clarity and predictability around risk assessment for informational functions while recognizing that directiveness may exist along a continuum rather than as a binary distinction?
In the summary report: Takeaway 4, A disclaimer does not make AI advice safer, filers told the FDA
Answer chart: of the 75 filers who took a side on this point (no 72, partially 2, yes 1). The answer chart counts only filers who named one option; proposal shares are of all who answered. Proposals: of the 198 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“A “talk to your doctor” disclaimer should receive no credit toward reducing directiveness.”Hippocratic AI The full filing →Clinicians · Judge by user effect
“ACS recommends that FDA judge whether a function is action-directing by its effect on the user, evaluated through human factors testing, rather than by its wording.”American College of Surgeons The full filing →Industry · Wants a mechanical test
“Use a mechanical test instead of a wording test: would the output be acted on by the device, or by a downstream system, absent a human decision?”Precision Health Ventures The full filing →
“A “talk to your doctor” disclaimer should receive no credit toward reducing directiveness.”Hippocratic AI The full filing →Industry · Wants worked examples
“We would treat this as a continuum and anchor it with worked examples, judging directiveness on substance as the paper proposes.”Bausch + Lomb The full filing →Clinicians · Judge by user effect
“ACS recommends that FDA judge whether a function is action-directing by its effect on the user, evaluated through human factors testing, rather than by its wording.”American College of Surgeons The full filing →
“ACS recommends that FDA judge whether a function is action-directing by its effect on the user, evaluated through human factors testing, rather than by its wording.”American College of Surgeons The full filing →Clinicians · Risk rises with personalization
“The classification of the device risk level should increase as outputs become more personalized, time-sensitive, clinically directive and likely to influence patient action.”Mayo Clinic The full filing →Industry · No credit for disclaimers
“A “talk to your doctor” disclaimer should receive no credit toward reducing directiveness.”Hippocratic AI The full filing →
“We support the approach CDRH is considering, in which directiveness depends on the substance and context of an output and not solely on its wording.”Intraoperative Performance Analytics Laboratory (IPAL) The full filing →Researchers · Disclaimers are not mitigation
“The FDA should not treat disclaimers as sufficient mitigation when assessing whether a response is action-directing, and should require evidence that intended users understand the …”NSF AI Research Institute on Interaction for AI Assistants (ARIA) The full filing →Industry · No credit for disclaimers
“A “talk to your doctor” disclaimer should receive no credit toward reducing directiveness.”Hippocratic AI The full filing →
“A disclaimer should not change what a product is designed and distributed to do in neither the risk assessment nor intended use analyses.”Tech Justice Law The full filing →Industry · No credit for disclaimers
“A “talk to your doctor” disclaimer should receive no credit toward reducing directiveness.”Hippocratic AI The full filing →Clinicians · Judge by user effect
“ACS recommends that FDA judge whether a function is action-directing by its effect on the user, evaluated through human factors testing, rather than by its wording.”American College of Surgeons The full filing →
“An informational function may become action-directing through personalization, repetition, omission of alternatives, urgency, authority cues, or accumulation across turns.”VivaSecuris The full filing →Other · Judge by observed behavior
“Directiveness should be judged by observed behavior, not interface labels.”VitaSignal The full filing →Industry · No credit for disclaimers
“A “talk to your doctor” disclaimer should receive no credit toward reducing directiveness.”Hippocratic AI The full filing →
Only 13 filers in this group, so counts are shown, not shares.
Only 3 filers in this group, so counts are shown, not shares.
Only 17 filers who took a side in this group, so counts are shown, not shares.
Only 5 filers who took a side in this group, so counts are shown, not shares.
Only 2 filers who took a side in this group, so counts are shown, not shares.
Only 10 filers who took a side in this group, so counts are shown, not shares.
CDRH seeks input on whether and when a GenAI-enabled function that results in delivery of clinical information to patients, as opposed to HCPs, could present different or higher risks, while also recognizing the potential benefits associated with improved patient empowerment, engagement, and access to clinical information. What device characteristics, output features, or safeguards might mitigate risks that could arise when a user lacks the domain knowledge to independently evaluate an output, without unnecessarily underestimating patient capability?
In the summary report: Takeaway 4, A disclaimer does not make AI advice safer, filers told the FDA
Answer chart: of the 202 filers who answered (depends 88, not riskier 36, always riskier 12, no side 66). Proposals: of the 202 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Patient-facing delivery should affect the risk assessment, but it should not determine the risk category by itself.”Clara Health The full filing →Clinicians · Higher risk by default
“Patient-facing functions should be treated as higher risk by default.”Doctors for America The full filing →Other · Not inherently riskier
“We do not consider patient-facing information to present inherently higher risk.”Scarlet The full filing →
“Patient-facing delivery should affect the risk assessment, but it should not determine the risk category by itself.”Clara Health The full filing →Industry · Different, not higher risk
“Patient-facing AI may carry different risks, but not automatically higher ones.”Lune Health The full filing →Clinicians · Higher risk by default
“Patient-facing functions should be treated as higher risk by default.”Doctors for America The full filing →
“Patient-facing functions may create different risks because a patient may act without clinical intermediation, but they should not be presumed higher risk in every case.”Johns Hopkins Medicine The full filing →Clinicians · Wants rigorous premarket review
“Direct-to-consumer and other patient-facing tools warrant particularly rigorous premarket evaluation because patients may act on outputs without immediate clinician interpretation.”American College of Cardiology The full filing →Industry · A factor, not decisive
“Patient-facing delivery should affect the risk assessment, but it should not determine the risk category by itself.”Clara Health The full filing →
“We agree with the paper’s premise that patients, lacking clinical training, may be less able than HCPs to recognize an incorrect or subtly misaligned output and to avoid relying on it.”Yale Collaboration for Regulatory Rigor Integrity and Transparency (CRRIT) The full filing →Researchers · Access is not risk
“Patient access should not itself be treated as a risk; safeguards should be proportionate to a device’s intended function and potential harm.”Melinda Rubinstein The full filing →Industry · A factor, not decisive
“Patient-facing delivery should affect the risk assessment, but it should not determine the risk category by itself.”Clara Health The full filing →
“Patient-facing use or disability should not, by itself, determine that a device presents greater risk.”The ALS Association The full filing →Patients · No categorical higher risk
“FDA’s framework should therefore avoid treating patient-facing use as categorically higher risk based solely on the intended user’s professional credentials.”National Health Council The full filing →Industry · A factor, not decisive
“Patient-facing delivery should affect the risk assessment, but it should not determine the risk category by itself.”Clara Health The full filing →
“Patient-facing use should be considered a contextual risk modifier rather than an automatic risk category.”HL7 Caliper Accelerator The full filing →Other · Not inherently riskier
“We do not consider patient-facing information to present inherently higher risk.”Scarlet The full filing →Industry · A factor, not decisive
“Patient-facing delivery should affect the risk assessment, but it should not determine the risk category by itself.”Clara Health The full filing →
Only 10 filers in this group, so counts are shown, not shares.
Only 4 filers in this group, so counts are shown, not shares.
Only 10 filers in this group, so counts are shown, not shares.
Only 4 filers in this group, so counts are shown, not shares.
CDRH seeks input on whether and how the distinction between generalist and specialist physicians might inform the assessment of risk for HCP-facing GenAI-enabled functions. Under what circumstances, if any, might risk be affected when an HCP who lacks the relevant clinical specialist knowledge receives information that falls within a specialist area of practice? What device characteristics or safeguards might mitigate such risks?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 178 specific asks linked to Q4 (more oversight 81, mixed 8, less oversight 11, neither: method or clarity 78). Proposals: of the 142 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Mayo Clinic recommends that FDA assess risk based on the relationship between the complexity of the task and the competency of the intended user, rather than relying on whether …”Mayo Clinic The full filing →Industry · Generalist use not riskier
“AIHC also cautions against treating use by generalist clinicians as a risk-elevating factor.”AI Healthcare Coalition The full filing →Other · Generalist use raises risk
“Risk escalates significantly when specialist recommendations are delivered to generalist clinicians who lack the subspecialty depth to detect subtle hallucinations.”AI Nova Strategy (Kevin J. Hacker) The full filing →
“It should, but not as a rule that generalist use is inherently riskier. Risk rises when the safe use of a finding depends on specialty expertise the intended user lacks.”Veritas Biomedical Analytics, Inc. The full filing →Industry · Generalist use not riskier
“AIHC also cautions against treating use by generalist clinicians as a risk-elevating factor.”AI Healthcare Coalition The full filing →Clinicians · Match task to competence
“Mayo Clinic recommends that FDA assess risk based on the relationship between the complexity of the task and the competency of the intended user, rather than relying on whether …”Mayo Clinic The full filing →
“Mayo Clinic recommends that FDA assess risk based on the relationship between the complexity of the task and the competency of the intended user, rather than relying on whether …”Mayo Clinic The full filing →Clinicians · Include sonographers and others
“SDMS recommends that FDA include sonographers and other relevant healthcare professionals in its discussion of generalist and specialist users.”Society of Diagnostic Medical Sonography The full filing →Industry · Generalist use not riskier
“AIHC also cautions against treating use by generalist clinicians as a risk-elevating factor.”AI Healthcare Coalition The full filing →
“Yes, although in surgery the relevant distinction is experience more than specialty.”Intraoperative Performance Analytics Laboratory (IPAL) The full filing →Researchers · Distinction may be relevant
“I agree that the distinction between generalist and specialist physicians may be relevant for Criticality Stratification.”Martin Haimerl The full filing →Clinicians · Match task to competence
“Mayo Clinic recommends that FDA assess risk based on the relationship between the complexity of the task and the competency of the intended user, rather than relying on whether …”Mayo Clinic The full filing →
“FDA should consider the knowledge reasonably available to the intended user, the clinical consequences of error, and whether the device provides adequate context and pathways for …”National Health Council The full filing →Clinicians · Match task to competence
“Mayo Clinic recommends that FDA assess risk based on the relationship between the complexity of the task and the competency of the intended user, rather than relying on whether …”Mayo Clinic The full filing →Industry · Generalist use not riskier
“AIHC also cautions against treating use by generalist clinicians as a risk-elevating factor.”AI Healthcare Coalition The full filing →
“The relevant distinction is not whether an HCP is a generalist or a specialist, but whether they have the expertise to critically assess a particular output.”Scarlet The full filing →Other · Depends on needed expertise
“The relevant consideration should be whether safe use depends on specialized knowledge necessary to independently evaluate the device output.”HL7 Caliper Accelerator The full filing →Clinicians · Match task to competence
“Mayo Clinic recommends that FDA assess risk based on the relationship between the complexity of the task and the competency of the intended user, rather than relying on whether …”Mayo Clinic The full filing →
Only 5 filers in this group, so counts are shown, not shares.
Only 1 filer in this group, so counts are shown, not shares.
Only 6 asks in this group, so counts are shown, not shares.
Only 1 asks in this group, so counts are shown, not shares.
For multi-turn conversational GenAI-enabled devices that may migrate from providing “non-directive” information to “action-directing” information over the course of an exchange, how should risk be assessed across realistic conversational trajectories? How could the intended use of such a device be characterized when its behavior is emergent across a conversation?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 261 specific asks linked to Q5 (more oversight 144, mixed 4, less oversight 5, neither: method or clarity 108). Proposals: of the 189 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Assess risk at the level of the conversation, not the turn.”David Liebovitz The full filing →Clinicians · Rate by riskiest state
“For multi-turn conversational systems, risk should be based on the highest-risk state the device can reasonably reach across a foreseeable interaction—not only its initial response.”American Society for Gastrointestinal Endoscopy The full filing →Other · Coaxed outputs don’t define use
“CHI recommends that FDA not define a device’s intended use by the most directive output a user could elicit through persistent or adversarial prompting”Connected Health Initiative The full filing →
“Intended use should be defined by the bounded set of outputs and actions the device is permitted to produce.”Grasshopper Health, Inc. The full filing →Industry · Test pushy, long conversations
“Risk should be assessed across realistic conversations, including long exchanges, users in distress, young users, and deliberate attempts to push a system beyond its scope.”Kooth The full filing →Researchers · Judge the whole conversation
“Assess risk at the level of the conversation, not the turn.”David Liebovitz The full filing →
“For multi-turn conversational systems, risk should be based on the highest-risk state the device can reasonably reach across a foreseeable interaction—not only its initial response.”American Society for Gastrointestinal Endoscopy The full filing →Clinicians · Judge the whole encounter
“FDA should evaluate the entire encounter, not individual prompt-response pairs.”Deborah Ault, RN The full filing →Researchers · Judge the whole conversation
“Assess risk at the level of the conversation, not the turn.”David Liebovitz The full filing →
“Assess risk at the level of the conversation, not the turn.”David Liebovitz The full filing →Researchers · Assess path to recommendation
“For conversations, assess the path from general information to a personalised recommendation or action, including prohibited transitions.”Yubin Kim The full filing →Clinicians · Rate by riskiest state
“For multi-turn conversational systems, risk should be based on the highest-risk state the device can reasonably reach across a foreseeable interaction—not only its initial response.”American Society for Gastrointestinal Endoscopy The full filing →
“The NHC agrees that GenAI risk cannot always be assessed one interaction at a time.”National Health Council The full filing →Patients · Rate by riskiest function
“Generation Patient asks CDRH to evaluate risk of conversational AI devices based on the most directive function they can perform and its most severe consequences, regardless of …”Generation Patient The full filing →Researchers · Judge the whole conversation
“Assess risk at the level of the conversation, not the turn.”David Liebovitz The full filing →
“Risk for multi-turn devices should be assessed at the level of the trajectory, not the single output.”An individual (unknown) The full filing →Other · Coaxed outputs don’t define use
“CHI recommends that FDA not define a device’s intended use by the most directive output a user could elicit through persistent or adversarial prompting”Connected Health Initiative The full filing →Researchers · Judge the whole conversation
“Assess risk at the level of the conversation, not the turn.”David Liebovitz The full filing →
Only 16 filers in this group, so counts are shown, not shares.
Only 3 filers in this group, so counts are shown, not shares.
Only 18 asks in this group, so counts are shown, not shares.
Only 3 asks in this group, so counts are shown, not shares.
For GenAI-enabled care escalation functions, CDRH is considering how the evaluation may account for both under-escalation and over-escalation. How could manufacturers characterize and weigh these two directions of error, given that they may not be commensurable and that acceptable trade-offs may vary by clinical context?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Answer chart: of the 119 filers who took a side on this point (by clinical context 56, report each separately 41, missed escalation weighs more 17, more than two error types 5). The answer chart counts only filers who named one option; proposal shares are of all who answered. Proposals: of the 173 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Under-escalation and over-escalation should not be combined into a single score. They should be measured separately, each with its own ceiling, and the acceptable ratio between …”Sudha Jayaraman The full filing →Clinicians · Never one accuracy figure
“Under- and over-escalation are not commensurable and should never be summarized by a single accuracy figure.”Neel N. Patel The full filing →Other · Missed escalation matters most
“Under-escalation is the primary risk and should be the focus of evaluation.”Curai Health The full filing →
“Under-escalation and over-escalation should not be combined into a single score. They should be measured separately, each with its own ceiling, and the acceptable ratio between …”Sudha Jayaraman The full filing →Industry · Set weights before testing
“Their relative weight should be set per clinical context, before testing, with clinicians and patients involved.”Implementation Alpha AI The full filing →Clinicians · Never one accuracy figure
“Under- and over-escalation are not commensurable and should never be summarized by a single accuracy figure.”Neel N. Patel The full filing →
“Under- and over-escalation are not commensurable and should never be summarized by a single accuracy figure.”Neel N. Patel The full filing →Clinicians · Weigh by time sensitivity
“ACS agrees that the two directions of error are not commensurable and recommends weighing them by the time sensitivity of the condition that could be missed.”American College of Surgeons The full filing →Industry · Separate, weighed by context
“Under-escalation and over-escalation should not be combined into a single score. They should be measured separately, each with its own ceiling, and the acceptable ratio between …”Sudha Jayaraman The full filing →
“Report under-escalation and over-escalation separately.”Yubin Kim The full filing →Researchers · Over-escalation is a harm
“Another strength of the framework is the recognition that over-escalation to care is a harm worthy of consideration.”Stanford Institute for Human-Centered Artificial Intelligence (HAI) The full filing →Industry · Separate, weighed by context
“Under-escalation and over-escalation should not be combined into a single score. They should be measured separately, each with its own ceiling, and the acceptable ratio between …”Sudha Jayaraman The full filing →
“The NHC encourages FDA to allow these errors to be weighed differently depending on clinical context rather than relying on a single undifferentiated measure of performance.”National Health Council The full filing →Industry · Separate, weighed by context
“Under-escalation and over-escalation should not be combined into a single score. They should be measured separately, each with its own ceiling, and the acceptable ratio between …”Sudha Jayaraman The full filing →Clinicians · Never one accuracy figure
“Under- and over-escalation are not commensurable and should never be summarized by a single accuracy figure.”Neel N. Patel The full filing →
“Measure both under-escalation and over-escalation using separate clinically justified thresholds; do not collapse them into a single accuracy score.”VivaSecuris The full filing →Other · Missed escalation matters most
“Under-escalation is the primary risk and should be the focus of evaluation.”Curai Health The full filing →Industry · Separate, weighed by context
“Under-escalation and over-escalation should not be combined into a single score. They should be measured separately, each with its own ceiling, and the acceptable ratio between …”Sudha Jayaraman The full filing →
Only 14 filers in this group, so counts are shown, not shares.
Only 1 filer in this group, so counts are shown, not shares.
Only 9 filers who took a side in this group, so counts are shown, not shares.
Only 1 filer who took a side in this group, so counts are shown, not shares.
Only 15 filers who took a side in this group, so counts are shown, not shares.
Is the competency-based approach described above, i.e., device benchmarking followed by clinical confirmation, a useful and appropriate framework for evaluating GenAI-enabled devices?
In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes
Answer chart: of the 273 filers who answered (accept, with changes 134, accept, as written 57, split 1, reject 3, no verdict 78). Proposals: of the 273 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“The competency approach is useful and necessary. The manufacturer must test the final device configuration. …”Newton’s Tree Ltd. The full filing →Other · Add scheduled re-benchmarking
“The competency-assessment approach — benchmark, then clinical confirmation, then scheduled re-benchmarking with independent review of real input/output samples — is the right shape.”An individual (unknown) The full filing →Clinicians · Rejects it as framed
“No. The DPA does not support the competency-based approach as framed in the discussion paper, for three reasons:”Digital Pathology Association The full filing →
“The competency approach is useful and necessary. The manufacturer must test the final device configuration. …”Newton’s Tree Ltd. The full filing →Industry · Backs the two steps
“Yes. Competency benchmarking followed by clinical confirmation is the right framework, and it reflects how mature clinical AI teams already evaluate.”Counsel Health The full filing →Other · Add scheduled re-benchmarking
“The competency-assessment approach — benchmark, then clinical confirmation, then scheduled re-benchmarking with independent review of real input/output samples — is the right shape.”An individual (unknown) The full filing →
“Mayo Clinic supports the general structure of a competency-based approach that combines device benchmarking with clinical confirmation.”Mayo Clinic The full filing →Clinicians · Benchmarks cannot replace validation
“AMIA supports competency-based evaluation as a useful component of FDA review, but benchmarking should complement, not replace, clinical validation.”American Medical Informatics Association The full filing →Industry · Test the final device
“The competency approach is useful and necessary. The manufacturer must test the final device configuration. …”Newton’s Tree Ltd. The full filing →
“I support the proposed approach of benchmarking the final device followed by clinical confirmation.”Binesh Sadanandan The full filing →Researchers · Wants real-use clinical evidence
“The FDA should require clinical evidence that reflects how the device will be used.”Program On Regulation, Therapeutics, And Law (PORTAL) and MIT Critical Data The full filing →Industry · Test the final device
“The competency approach is useful and necessary. The manufacturer must test the final device configuration. …”Newton’s Tree Ltd. The full filing →
“The NHC supports FDA’s exploration of a competency-based approach that combines device benchmarking with clinical confirmation.”National Health Council The full filing →Industry · Test the final device
“The competency approach is useful and necessary. The manufacturer must test the final device configuration. …”Newton’s Tree Ltd. The full filing →Other · Add scheduled re-benchmarking
“The competency-assessment approach — benchmark, then clinical confirmation, then scheduled re-benchmarking with independent review of real input/output samples — is the right shape.”An individual (unknown) The full filing →
“Yes. We consider the competency-based approach a useful and appropriate framework for evaluating GenAI-enabled devices.”Scarlet The full filing →Other · Add scheduled re-benchmarking
“The competency-assessment approach — benchmark, then clinical confirmation, then scheduled re-benchmarking with independent review of real input/output samples — is the right shape.”An individual (unknown) The full filing →Industry · Test the final device
“The competency approach is useful and necessary. The manufacturer must test the final device configuration. …”Newton’s Tree Ltd. The full filing →
Only 3 filers in this group, so counts are shown, not shares.
Of the 195 who gave a verdict, 134 accepted with changes, 57 accepted as written, 1 was split and 3 rejected.
Of the 106 who gave a verdict, 72 accepted with changes, 31 accepted as written, 1 was split and 2 rejected.
Of the 43 who gave a verdict, 34 accepted with changes, 8 accepted as written and 1 rejected.
Of the 18 who gave a verdict, 12 accepted with changes and 6 accepted as written.
The one filer who gave a verdict accepted as written.
Of the 27 who gave a verdict, 16 accepted with changes and 11 accepted as written.
Only 3 filers in this group, so counts are shown, not shares.
How might the two-axis risk framework described in Section IV be considered within the competency-based approach described above to help determine the level of evidence needed for a premarket submission?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 225 specific asks linked to Q8 (more oversight 80, mixed 17, less oversight 21, neither: method or clarity 107). Proposals: of the 176 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“We ask that the two-axis risk framework directly scale both the breadth of benchmarking required and how far along the clinical confirmation spectrum a sponsor must go.”American College of Allergy, Asthma and Immunology The full filing →Industry · Wants an evidence matrix
“Danaher recommends that FDA publish an illustrative matrix mapping each combination of Activity and Consequence to expected evidence elements.”Danaher Corporation The full filing →Industry · Hazard analysis sets evidence
“The two-axis framework should not directly set the amount of evidence. The hazard analysis and the effectiveness of the risk controls should set the evidence requirements.”Newton’s Tree Ltd. The full filing →
“Specifically, higher device activity and/or higher output consequence should carry a higher burden of evidence.”Mosaic Intelligence Labs, Inc. The full filing →Industry · Wants an evidence matrix
“Danaher recommends that FDA publish an illustrative matrix mapping each combination of Activity and Consequence to expected evidence elements.”Danaher Corporation The full filing →Clinicians · Axes scale evidence depth
“We ask that the two-axis risk framework directly scale both the breadth of benchmarking required and how far along the clinical confirmation spectrum a sponsor must go.”American College of Allergy, Asthma and Immunology The full filing →
“We ask that the two-axis risk framework directly scale both the breadth of benchmarking required and how far along the clinical confirmation spectrum a sponsor must go.”American College of Allergy, Asthma and Immunology The full filing →Clinicians · Credits proven safeguards only
“Scale evidence to consequence, autonomy, detectability, and available safeguards; a supervision label should reduce evidence only when its effectiveness is demonstrated.”American Association of Physicists in Medicine The full filing →Industry · Wants an evidence matrix
“Danaher recommends that FDA publish an illustrative matrix mapping each combination of Activity and Consequence to expected evidence elements.”Danaher Corporation The full filing →
“The depth of each should be proportionate to the autonomy of the device and the consequence of an incorrect decision.”Clinical Excellence Research Center, Stanford University (Aditya Narayan) The full filing →Researchers · Every evidence type required
“The depth of each type of evidence may reasonably scale with risk, but none should be absent.”Yale Collaboration for Regulatory Rigor Integrity and Transparency (CRRIT) The full filing →Clinicians · Axes scale evidence depth
“We ask that the two-axis risk framework directly scale both the breadth of benchmarking required and how far along the clinical confirmation spectrum a sponsor must go.”American College of Allergy, Asthma and Immunology The full filing →
“The ALS Association supports rigorous evidence requirements appropriate to a device’s intended use, patient population, level of autonomy, and the potential consequences of an …”The ALS Association The full filing →Clinicians · Axes scale evidence depth
“We ask that the two-axis risk framework directly scale both the breadth of benchmarking required and how far along the clinical confirmation spectrum a sponsor must go.”American College of Allergy, Asthma and Immunology The full filing →Industry · Wants an evidence matrix
“Danaher recommends that FDA publish an illustrative matrix mapping each combination of Activity and Consequence to expected evidence elements.”Danaher Corporation The full filing →
“Higher-risk devices should require more rigorous benchmarking, representative testing, and clinical confirmation.”Digital Medicine Society (DiMe) The full filing →Other · Safeguards must prove effective
“The appropriate level of evidence should be determined by the function's clinical risk and the effectiveness of its safeguards, rather than the presence of clinician supervision alone.”Alliance for Connected Care The full filing →Clinicians · Axes scale evidence depth
“We ask that the two-axis risk framework directly scale both the breadth of benchmarking required and how far along the clinical confirmation spectrum a sponsor must go.”American College of Allergy, Asthma and Immunology The full filing →
Only 15 filers in this group, so counts are shown, not shares.
Only 2 filers in this group, so counts are shown, not shares.
Only 12 asks in this group, so counts are shown, not shares.
Only 1 asks in this group, so counts are shown, not shares.
Would a benchmarking structure such as the one described above be likely to provide adequate evidence of clinical knowledge, analytic capabilities, safety behavior, communication, and generalizability to support a reasonable assurance of safety and effectiveness? Are there elements that are missing, redundant, or inappropriately categorized? Are there externally developed standards that could be leveraged?
In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes
Answer chart: of the 255 filers who answered (accept, with changes 61, accept, as written 3, split 0, reject 0, no verdict 191). Proposals: of the 255 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“The proposed benchmarking structure is a useful foundation, but it is unlikely to provide sufficient evidence of safety and effectiveness on its own.”American Academy of Family Physicians The full filing →Industry · Elements as a menu
“In particular, ACRO recommends that CDRH clarify in future communications that (1) benchmarking elements are a menu from which sponsors select based on intended use and risk;”Association of Clinical Research Organizations (ACRO) The full filing →Researchers · Wants subgroup equity reporting
“Benchmarking should include a language-and-equity competency, reported by subgroup.”David Liebovitz The full filing →
“The proposed benchmarking structure maps closely onto the evaluation suite we run, and we think it is sound.”Counsel Health The full filing →Industry · Adds adversarial edge cases
“Q9: the Appendix A elements are a good list; I would add adversarial and edge-case coverage as an explicit element”Patrick James The full filing →Clinicians · Useful but not sufficient
“The proposed benchmarking structure is a useful foundation, but it is unlikely to provide sufficient evidence of safety and effectiveness on its own.”American Academy of Family Physicians The full filing →
“The proposed benchmarking structure is a useful foundation, but it is unlikely to provide sufficient evidence of safety and effectiveness on its own.”American Academy of Family Physicians The full filing →Clinicians · Requires gender subgroup testing
“Gender should be an explicit, required dimension of subgroup evaluation.”Asma Khapra The full filing →Industry · Elements as a menu
“In particular, ACRO recommends that CDRH clarify in future communications that (1) benchmarking elements are a menu from which sponsors select based on intended use and risk;”Association of Clinical Research Organizations (ACRO) The full filing →
“Benchmarking should include a language-and-equity competency, reported by subgroup.”David Liebovitz The full filing →Researchers · Include neurodivergent users
“For devices likely to encounter neurodivergent patients, manufacturers should explain whether and how autistic, ADHD, and other relevant users were represented in evaluation data.”David Ruttenberg The full filing →Clinicians · Useful but not sufficient
“The proposed benchmarking structure is a useful foundation, but it is unlikely to provide sufficient evidence of safety and effectiveness on its own.”American Academy of Family Physicians The full filing →
“Testing medical devices across populations is appropriate and consistent with what Congress has directed.”Generation Patient The full filing →Patients · Accuracy alone falls short
“Technical accuracy alone may not fully describe whether a patient-facing GenAI-enabled medical device performs successfully.”The ALS Association The full filing →Clinicians · Useful but not sufficient
“The proposed benchmarking structure is a useful foundation, but it is unlikely to provide sufficient evidence of safety and effectiveness on its own.”American Academy of Family Physicians The full filing →
“FDA's ten benchmark elements are a strong core.”Alfred McBride The full filing →Other · Needs clinical confirmation too
“Since benchmarking is often not sufficient, it is important to use some form of clinical confirmation, proportionate to risk.”Bipartisan Policy Center The full filing →Clinicians · Useful but not sufficient
“The proposed benchmarking structure is a useful foundation, but it is unlikely to provide sufficient evidence of safety and effectiveness on its own.”American Academy of Family Physicians The full filing →
Only 3 filers in this group, so counts are shown, not shares.
Of the 64 who gave a verdict, 61 accepted with changes and 3 accepted as written.
Of the 34 who gave a verdict, 32 accepted with changes and 2 accepted as written.
Of the 18 who gave a verdict, 17 accepted with changes and 1 accepted as written.
Both who gave a verdict accepted with changes.
The one filer who gave a verdict accepted with changes.
All 9 who gave a verdict accepted with changes.
Only 3 filers in this group, so counts are shown, not shares.
CDRH recognizes that publicly available benchmarking assets may be subject to data contamination, saturation, and limited real-world representativeness. How should a sponsor establish that performance on a given benchmark predicts safe and effective real-world behavior for the device’s intended use? What evidence should support the construct validity of a benchmark used to gate device evaluation, and what role should sponsor-developed benchmarks play given potential concerns around independence and optimization to the test?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 371 specific asks linked to Q10 (more oversight 267, mixed 3, less oversight 4, neither: method or clarity 97). Proposals: of the 223 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Public benchmark performance should not be presumed to predict clinical performance.”VitaSignal The full filing →Clinicians · Wants sequestered test sets
“Sequestered test sets are the best response to the contamination and saturation described in Question 10, and sponsor-developed benchmarks invite optimization to the test.”Jesus Recio The full filing →Industry · Accepts sponsor-built benchmarks
“Sponsor-developed benchmarks should not be disfavored solely because they are sponsor-developed.”Baxter The full filing →
“We believe the best evidence that a benchmark is meaningful is showing that scores on it track separately measured performance of the same product in real use.”Slingshot AI, Inc. The full filing →Industry · Sponsor tests, if sequestered
“Sponsor-developed benchmarks should be acceptable if they are prespecified and sequestered before testing so they cannot be tuned to the test.”Veritas Biomedical Analytics, Inc. The full filing →Other · Prediction must be shown
“Public benchmark performance should not be presumed to predict clinical performance.”VitaSignal The full filing →
“Sequestered test sets are the best response to the contamination and saturation described in Question 10, and sponsor-developed benchmarks invite optimization to the test.”Jesus Recio The full filing →Clinicians · Must predict real inputs
“Sponsors should demonstrate that benchmark performance predicts behavior on real inputs from the intended population.”American Society for Gastrointestinal Endoscopy The full filing →Other · Prediction must be shown
“Public benchmark performance should not be presumed to predict clinical performance.”VitaSignal The full filing →
“Sponsors should validate benchmark performance using real-world clinical data and representative patient scenarios that reflect the device's intended use.”Loma Linda University School of Medicine (Harine Park) The full filing →Researchers · Independent sequestered test sets
“Use sequestered, multi-institutional test sets held independently of developers (Question 16).”Intraoperative Performance Analytics Laboratory (IPAL) The full filing →Other · Prediction must be shown
“Public benchmark performance should not be presumed to predict clinical performance.”VitaSignal The full filing →
“The NHC also encourages FDA clarify the characteristics of fit-for-purpose benchmarks used to support regulatory decisions.”National Health Council The full filing →Other · Prediction must be shown
“Public benchmark performance should not be presumed to predict clinical performance.”VitaSignal The full filing →Clinicians · Wants sequestered test sets
“Sequestered test sets are the best response to the contamination and saturation described in Question 10, and sponsor-developed benchmarks invite optimization to the test.”Jesus Recio The full filing →
“Public benchmark performance should not be presumed to predict clinical performance.”VitaSignal The full filing →Other · Distrusts sponsor-written tests
“When the sponsor writes the test, the test measures the sponsor.”World Privacy Forum The full filing →Clinicians · Wants sequestered test sets
“Sequestered test sets are the best response to the contamination and saturation described in Question 10, and sponsor-developed benchmarks invite optimization to the test.”Jesus Recio The full filing →
Only 1 filer in this group, so counts are shown, not shares.
Only 1 asks in this group, so counts are shown, not shares.
CDRH is considering that clinical confirmation for a GenAI-enabled device might not require a prospective clinical study in every case, and has described above a range of approaches of increasing rigor and patient exposure. How might a sponsor select and justify a confirmation approach tailored to a device’s intended use and proportionate to the device’s risk profile? Are there device types or risk profiles for which one or more of these approaches would be insufficient or inappropriate? How might the anticipated distribution of real-world inputs be taken into consideration? Are there other methods of clinical confirmation that might help inform the evaluation of GenAI-enabled devices?
In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes
Answer chart: of the 209 filers who answered (accept, with changes 57, accept, as written 13, split 1, reject 0, no verdict 138). Proposals: of the 209 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“We support FDA's proposal that clinical confirmation be proportionate to intended use and risk and not necessarily require a prospective clinical study.”QRx Partners The full filing →Clinicians · Irreversible harm needs more
“For procedural functions capable of immediate, irreversible harm, the less rigorous approaches are unlikely to be sufficient on their own.”American College of Surgeons The full filing →Researchers · Must test user response
“Retrospective or shadow studies may be suitable for some questions. They cannot by them- selves establish whether people will notice, trust or act on an alert displayed during care.”Yubin Kim The full filing →
“We agree that a prospective clinical study should not be required in every case.”GE HealthCare The full filing →Industry · Trials for highest risk
“We support a risk-based approach under which only the highest-risk cases require a prospective clinical study (e.g., those involving autonomous functions and severe consequences).”Oracle Health & Life Sciences The full filing →Other · Supports risk-proportionate confirmation
“We support FDA's proposal that clinical confirmation be proportionate to intended use and risk and not necessarily require a prospective clinical study.”QRx Partners The full filing →
“We support proportionality, but would set a floor.”American College of Allergy, Asthma and Immunology The full filing →Clinicians · Irreversible harm needs more
“For procedural functions capable of immediate, irreversible harm, the less rigorous approaches are unlikely to be sufficient on their own.”American College of Surgeons The full filing →Other · Supports risk-proportionate confirmation
“We support FDA's proposal that clinical confirmation be proportionate to intended use and risk and not necessarily require a prospective clinical study.”QRx Partners The full filing →
“Retrospective or shadow studies may be suitable for some questions. They cannot by them- selves establish whether people will notice, trust or act on an alert displayed during care.”Yubin Kim The full filing →Researchers · Test with intended users
“Benchmark results should be supplemented, proportionate to risk, by clinical or clinically representative evaluation involving the people and workflows for which a device is intended.”David Ruttenberg The full filing →Other · Supports risk-proportionate confirmation
“We support FDA's proposal that clinical confirmation be proportionate to intended use and risk and not necessarily require a prospective clinical study.”QRx Partners The full filing →
“Standardized testing alone is insufficient. Testing should occur with diverse, real users to ensure greater accuracy.”Tech Justice Law The full filing →Patients · Judge the whole package
“The central question should be whether the evidence package is sufficient for the intended use and clinical context rather than whether one study design is preferred in all cases.”National Health Council The full filing →Other · Supports risk-proportionate confirmation
“We support FDA's proposal that clinical confirmation be proportionate to intended use and risk and not necessarily require a prospective clinical study.”QRx Partners The full filing →
“We support FDA's proposal that clinical confirmation be proportionate to intended use and risk and not necessarily require a prospective clinical study.”QRx Partners The full filing →Other · Opposes trials for every device
“We oppose a universal requirement for randomized controlled trials.”Curai Health The full filing →Clinicians · Irreversible harm needs more
“For procedural functions capable of immediate, irreversible harm, the less rigorous approaches are unlikely to be sufficient on their own.”American College of Surgeons The full filing →
Only 19 filers in this group, so counts are shown, not shares.
Only 2 filers in this group, so counts are shown, not shares.
Of the 71 who gave a verdict, 57 accepted with changes, 13 accepted as written and 1 was split.
Of the 46 who gave a verdict, 34 accepted with changes, 11 accepted as written and 1 was split.
Of the 13 who gave a verdict, 12 accepted with changes and 1 accepted as written.
The one filer who gave a verdict accepted with changes.
The one filer who gave a verdict accepted with changes.
Of the 10 who gave a verdict, 9 accepted with changes and 1 accepted as written.
Only 19 filers in this group, so counts are shown, not shares.
Only 2 filers in this group, so counts are shown, not shares.
How should sponsors achieve statistically meaningful performance measurement for GenAI-enabled devices? Where synthetically generated inputs supplement real patient data, how should sponsors account for differences between the synthetic and real-world distributions when estimating performance, and under what conditions, if any, is it appropriate to combine benchmarking evidence and clinical confirmation evidence to support a single performance estimate?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 257 specific asks linked to Q12 (more oversight 142, mixed 1, less oversight 3, neither: method or clarity 111). Proposals: of the 154 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“FDA may wish to consider separate reporting of synthetic and real-world performance before allowing a combined estimate.”OneSource Solutions International The full filing →Clinicians · Report confidence intervals
“In addition, benchmarking results should be accompanied by appropriate measures of uncertainty, such as confidence intervals, to facilitate meaningful interpretation of device …”College of American Pathologists The full filing →Industry · Flexibility for small sponsors
“I would urge CDRH to add flexibility to this framework for small and early-stage sponsors, such as academic-affiliated development teams, which may not be able to support separate …”Tianrui Huang The full filing →
“Report benchmarking and clinical confirmation separately by default.”Sia Partners The full filing →Industry · Estimates with uncertainty
“Performance should be expressed as an estimate with an appropriate measure of uncertainty, not as a categorical competency determination.”Johnson & Johnson The full filing →Other · Separate synthetic, real results
“FDA may wish to consider separate reporting of synthetic and real-world performance before allowing a combined estimate.”OneSource Solutions International The full filing →
“In addition, benchmarking results should be accompanied by appropriate measures of uncertainty, such as confidence intervals, to facilitate meaningful interpretation of device …”College of American Pathologists The full filing →Clinicians · Evaluate data sources separately
“Accordingly, FDA should require that the sponsor separately evaluate performance on real-world and synthetic data.”American College of Radiology (ACR) The full filing →Other · Separate synthetic, real results
“FDA may wish to consider separate reporting of synthetic and real-world performance before allowing a combined estimate.”OneSource Solutions International The full filing →
“We recommend that results from synthetic and real-patient data be analyzed and reported separately.”Duke-Margolis Institute for Health Policy The full filing →Researchers · Validate synthetic data signal
“We suggest that justification should not rest on statistical similarity alone, but include analytical validations comparing the predictive signal of the synthetic dataset.”DNV AS (Serena Elizabeth Marshall) The full filing →Other · Separate synthetic, real results
“FDA may wish to consider separate reporting of synthetic and real-world performance before allowing a combined estimate.”OneSource Solutions International The full filing →
“Sponsors should prespecify endpoints and analysis plans and provide sufficient evidence to characterize performance where failures could materially affect patients.”National Health Council The full filing →Other · Separate synthetic, real results
“FDA may wish to consider separate reporting of synthetic and real-world performance before allowing a combined estimate.”OneSource Solutions International The full filing →Clinicians · Report confidence intervals
“In addition, benchmarking results should be accompanied by appropriate measures of uncertainty, such as confidence intervals, to facilitate meaningful interpretation of device …”College of American Pathologists The full filing →
“FDA may wish to consider separate reporting of synthetic and real-world performance before allowing a combined estimate.”OneSource Solutions International The full filing →Other · Lower bound must pass
“CDRH should require a confidence interval for any accuracy metrics, 11 where the lower bound of the confidence interval is required to meet a passing threshold.”The Upstream Evidence Project The full filing →Clinicians · Report confidence intervals
“In addition, benchmarking results should be accompanied by appropriate measures of uncertainty, such as confidence intervals, to facilitate meaningful interpretation of device …”College of American Pathologists The full filing →
Only 1 filer in this group, so counts are shown, not shares.
Only 3 asks in this group, so counts are shown, not shares.
For which clinical domains, device functions, or subpopulations is synthetic data particularly well-suited, or particularly inadequate, as a supplement to real-world evidence? What safeguards would mitigate the risk that synthetic data generated by models of the same class as the device under evaluation reproduces the very performance gaps the evaluation is intended to detect, particularly for underrepresented subgroups?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 186 specific asks linked to Q13 (more oversight 139, mixed 1, less oversight 2, neither: method or clarity 44). Proposals: of the 164 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Synthetic data is well suited to stress-testing rare presentations, safety-critical edge cases, and adversarial inputs, and poorly suited as the sole basis for subgroup …”Visium The full filing →Clinicians · Supplement, not replace
“Synthetic cases may supplement, but not replace, representative real examinations.”Francis Deng The full filing →Researchers · Avoid same-family generators
“Synthetic data. Test inputs should not be generated by a model of the same family as the device under test, because shared blind spots will not be exposed.”Javid Rahaman The full filing →
“Synthetic data is well suited to stress-testing rare presentations, safety-critical edge cases, and adversarial inputs, and poorly suited as the sole basis for subgroup …”Visium The full filing →Industry · Generator from another family
“Synthetic data generators should be independent of the device's own model family, so that they do not reproduce its blind spots.”Edge Compliance The full filing →Clinicians · Supplement, not replace
“Synthetic cases may supplement, but not replace, representative real examinations.”Francis Deng The full filing →
“Synthetic cases may supplement, but not replace, representative real examinations.”Francis Deng The full filing →Clinicians · Useful for rare conditions
“Synthetic data may be useful for supplementing evaluation of rare conditions or scenarios that are difficult to observe at sufficient scale.”American Academy of Family Physicians The full filing →Industry · Stress tests; weak alone for subgroups
“Synthetic data is well suited to stress-testing rare presentations, safety-critical edge cases, and adversarial inputs, and poorly suited as the sole basis for subgroup …”Visium The full filing →
“Synthetic data. Test inputs should not be generated by a model of the same family as the device under test, because shared blind spots will not be exposed.”Javid Rahaman The full filing →Researchers · Can’t replace real subgroup data
“In response to Question 13, synthetic data may usefully expand scenario coverage, but it should not substitute for evidence in the real subgroups for whom performance is in question.”Yale Collaboration for Regulatory Rigor Integrity and Transparency (CRRIT) The full filing →Industry · Stress tests; weak alone for subgroups
“Synthetic data is well suited to stress-testing rare presentations, safety-critical edge cases, and adversarial inputs, and poorly suited as the sole basis for subgroup …”Visium The full filing →
“Synthetic data can be a useful supplement but should not routinely replace evidence from real patients and real-world settings.”National Health Council The full filing →Industry · Stress tests; weak alone for subgroups
“Synthetic data is well suited to stress-testing rare presentations, safety-critical edge cases, and adversarial inputs, and poorly suited as the sole basis for subgroup …”Visium The full filing →Clinicians · Supplement, not replace
“Synthetic cases may supplement, but not replace, representative real examinations.”Francis Deng The full filing →
“Synthetic data can help test rare cases but should supplement real-world data, not replace it.”ATA Action The full filing →Other · Document data origin
“Where synthetically generated inputs supplement real patient data, the analyses contemplated in Questions 12 and 13 benefit from explicit documentation of data origin.”Princeton Medical Systems The full filing →Industry · Stress tests; weak alone for subgroups
“Synthetic data is well suited to stress-testing rare presentations, safety-critical edge cases, and adversarial inputs, and poorly suited as the sole basis for subgroup …”Visium The full filing →
Only 17 filers in this group, so counts are shown, not shares.
Only 2 filers in this group, so counts are shown, not shares.
Only 17 asks in this group, so counts are shown, not shares.
Only 1 asks in this group, so counts are shown, not shares.
For open-ended device outputs, how should performance comparators and acceptance criteria be selected? When a panel of qualified clinicians serves as the comparator, how should the applicable standard (for example, the standard of care versus the performance of a median clinician in practice) be defined and justified? Should generalist or specialist physicians be used as a performance standard? When should human-AI team performance, rather than the device operating alone, serve as the basis for evaluation?
In the summary report: Takeaway 9, No single standard emerged for measuring AI devices
Answer chart: of the 129 filers who took a side on this point (tiered or several 46, match the real workflow 34, care without the device 25, specialist panel 11, evidence-based standard 11, median clinician 2). The answer chart counts only filers who named one option; proposal shares are of all who answered. Proposals: of the 241 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Depending on the claim, the relevant comparator may be qualified clinicians, standard care, accepted technology, or a human-AI team.”Johnson & Johnson The full filing →Clinicians · Prefers evidence-based standard
“The applicable comparator should generally reflect an evidence-based, clinically appropriate standard of care rather than median clinician performance, which may reflect unsafe or …”American Academy of Family Physicians The full filing →Other · Median clinician as comparator
“The comparator should be the median clinician in practice, which represents what an average physician would do.”Curai Health The full filing →
“Depending on the claim, the relevant comparator may be qualified clinicians, standard care, accepted technology, or a human-AI team.”Johnson & Johnson The full filing →Industry · Team versus clinician alone
“Where a clinician uses the device, the primary comparison should be human–AI team performance against the clinician working alone.”Implementation Alpha AI The full filing →Clinicians · Prefers evidence-based standard
“The applicable comparator should generally reflect an evidence-based, clinically appropriate standard of care rather than median clinician performance, which may reflect unsafe or …”American Academy of Family Physicians The full filing →
“The applicable comparator should generally reflect an evidence-based, clinically appropriate standard of care rather than median clinician performance, which may reflect unsafe or …”American Academy of Family Physicians The full filing →Clinicians · Test device and team
“FDA should assess both device-alone performance and human-AI team performance when the intended use depends on clinician interaction.”Radiological Society of North America The full filing →Industry · Comparator depends on claim
“Depending on the claim, the relevant comparator may be qualified clinicians, standard care, accepted technology, or a human-AI team.”Johnson & Johnson The full filing →
“The unit of analysis for clinical confirmation should presumptively be the human–AI team, compared against care without the device.”David Liebovitz The full filing →Researchers · Evidence-based care as reference
“The medical reference should be physician- adjudicated, evidence-based care—not a model’s ability to impersonate an average physician on an exam.”Clinical Excellence Research Center, Stanford University (Aditya Narayan) The full filing →Industry · Comparator depends on claim
“Depending on the claim, the relevant comparator may be qualified clinicians, standard care, accepted technology, or a human-AI team.”Johnson & Johnson The full filing →
“FDA should preserve flexibility while asking sponsors to justify why the selected comparator reflects the intended use and decision environment.”National Health Council The full filing →Industry · Comparator depends on claim
“Depending on the claim, the relevant comparator may be qualified clinicians, standard care, accepted technology, or a human-AI team.”Johnson & Johnson The full filing →Clinicians · Prefers evidence-based standard
“The applicable comparator should generally reflect an evidence-based, clinically appropriate standard of care rather than median clinician performance, which may reflect unsafe or …”American Academy of Family Physicians The full filing →
“The comparator should match the device's intended clinical role.”OneSource Solutions International The full filing →Other · Evaluate the clinician-AI team
“For technologies intended to augment clinical judgment, evaluation should consider the performance of the clinician-AI team within the intended workflow, rather than the AI system …”Alliance for Connected Care The full filing →Industry · Comparator depends on claim
“Depending on the claim, the relevant comparator may be qualified clinicians, standard care, accepted technology, or a human-AI team.”Johnson & Johnson The full filing →
Only 1 filer in this group, so counts are shown, not shares.
Only 11 filers who took a side in this group, so counts are shown, not shares.
No one in this group answered this part.
Only 14 filers who took a side in this group, so counts are shown, not shares.
Are there ways in which performance might be assessed relative to the care, technology, or course of action likely to occur in the absence of the device, rather than to the comparators described in this section? What approaches might be used to identify and justify a comparator such as unaided clinical judgment, delayed specialist review, or no intervention?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 151 specific asks linked to Q15 (more oversight 63, mixed 3, less oversight 8, neither: method or clarity 77). Proposals: of the 160 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“A realistic counterfactual — the care patients would receive without the device — can include delayed specialist review or limited access to care, but sponsors should justify that …”American Association of Physicists in Medicine The full filing →Industry · Allows realistic comparison
“FDA should allow comparison with the care that would realistically occur without the device.”OCHIN The full filing →Researchers · Absolute safety thresholds
“For safety-critical failures such as missed emergencies, acceptance thresholds should be absolute and should not be relaxed in exchange for benefit elsewhere.”Yale Collaboration for Regulatory Rigor Integrity and Transparency (CRRIT) The full filing →
“FDA should allow comparison with the care that would realistically occur without the device.”OCHIN The full filing →Industry · Not a safety criterion
“Comparison with the likely care pathway without the device may estimate clinical utility or incremental benefit, but it should not be a universal safety criterion.”Johnson & Johnson The full filing →Clinicians · Justify with setting evidence
“A realistic counterfactual — the care patients would receive without the device — can include delayed specialist review or limited access to care, but sponsors should justify that …”American Association of Physicists in Medicine The full filing →
“A realistic counterfactual — the care patients would receive without the device — can include delayed specialist review or limited access to care, but sponsors should justify that …”American Association of Physicists in Medicine The full filing →Clinicians · No substandard local baseline
“The comparator should reflect the care likely without the device, but a device should not be permitted to institutionalize substandard care merely because the local baseline is poor.”Douglas Stoddard, MD (CHRISTUS Health) The full filing →Industry · Allows realistic comparison
“FDA should allow comparison with the care that would realistically occur without the device.”OCHIN The full filing →
“For safety-critical failures such as missed emergencies, acceptance thresholds should be absolute and should not be relaxed in exchange for benefit elsewhere.”Yale Collaboration for Regulatory Rigor Integrity and Transparency (CRRIT) The full filing →Researchers · Typical care as comparator
“Typical care pathways, including realistic assessments of current patient population access to specialized expertise, may be appropriate comparators for tools intended to expand access.”Duke-Margolis Institute for Health Policy The full filing →Clinicians · Justify with setting evidence
“A realistic counterfactual — the care patients would receive without the device — can include delayed specialist review or limited access to care, but sponsors should justify that …”American Association of Physicists in Medicine The full filing →
“FDA should also consider the real-world alternative available to patients when evaluating potential benefit.”The ALS Association The full filing →Clinicians · Justify with setting evidence
“A realistic counterfactual — the care patients would receive without the device — can include delayed specialist review or limited access to care, but sponsors should justify that …”American Association of Physicists in Medicine The full filing →Industry · Allows realistic comparison
“FDA should allow comparison with the care that would realistically occur without the device.”OCHIN The full filing →
“Compare with the likely pathway without the device. This comparison is central to estimating real-world benefit and risk.”ARPA-H ADVOCATE Program; Applied Physics Lab at Johns Hopkins University The full filing →Other · Only no intervention works
“The only reliable comparator is the ‘no intervention’ condition. The others are too unpredictable and variable.”Ben Locwin The full filing →Clinicians · Justify with setting evidence
“A realistic counterfactual — the care patients would receive without the device — can include delayed specialist review or limited access to care, but sponsors should justify that …”American Association of Physicists in Medicine The full filing →
Only 14 filers in this group, so counts are shown, not shares.
Only 2 filers in this group, so counts are shown, not shares.
Only 12 asks in this group, so counts are shown, not shares.
Only 2 asks in this group, so counts are shown, not shares.
Should independent third parties be involved in some or all aspects of a competency-based approach, including device benchmarking and clinical confirmation? If so, in what ways might qualified, independent third-party participation contribute to this approach, and what qualifications and independence criteria should apply? Are there aspects of the assessment for which third-party involvement would be impractical or inadvisable? What safeguards or program design features would be critical to prevent third-party participation from limiting competition or preventing innovation?
In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes
Answer chart: of the 209 filers who answered (accept, with changes 65, accept, as written 3, split 3, reject 3, no verdict 135). Proposals: of the 209 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Yes, to maintain sequestered datasets and to serve as independent adjudicators. An independent party should have no financial interest in the sponsor or the foundation model developer.”Veritas Biomedical Analytics, Inc. The full filing →Clinicians · Backs third-party evaluation
“ACR supports the use of qualified, independent third parties for benchmarking, clinical confirmation, and postmarket assessment.”American College of Radiology (ACR) The full filing →Industry · Calls third parties redundant
“FDA’s existing oversight of AI-enabled medical devices renders third-party oversight of device performance redundant and impractical.”AdvaMed The full filing →
“Yes, in specific roles. We see the most value in maintaining sequestered test datasets, providing independent expert adjudication, and assessing benchmark validity.”Implementation Alpha AI The full filing →Industry · Keep third parties optional
“Independent third parties may provide useful datasets, evaluation tools, standards, or services, but their involvement should remain optional.”Epic The full filing →Industry · Holds data and adjudicates
“Yes, to maintain sequestered datasets and to serve as independent adjudicators. An independent party should have no financial interest in the sponsor or the foundation model developer.”Veritas Biomedical Analytics, Inc. The full filing →
“ACR supports the use of qualified, independent third parties for benchmarking, clinical confirmation, and postmarket assessment.”American College of Radiology (ACR) The full filing →Clinicians · No mandatory gatekeepers
“FDA should avoid requiring professional societies, certifiers, or other third parties to serve as mandatory gatekeepers.”American Gastroenterological Association The full filing →Industry · Holds data and adjudicates
“Yes, to maintain sequestered datasets and to serve as independent adjudicators. An independent party should have no financial interest in the sponsor or the foundation model developer.”Veritas Biomedical Analytics, Inc. The full filing →
“Maintaining sequestered datasets and supplying independent adjudication are the highest-value third-party roles.”Javid Rahaman The full filing →Researchers · Yes for higher-risk devices
“Yes, independent third parties should be involved in some of the benchmarking and clinical confirmation, especially in higher-risk GenAI-enabled medical devices.”Loma Linda University School of Medicine (Harine Park) The full filing →Industry · Holds data and adjudicates
“Yes, to maintain sequestered datasets and to serve as independent adjudicators. An independent party should have no financial interest in the sponsor or the foundation model developer.”Veritas Biomedical Analytics, Inc. The full filing →
“The NHC encourages FDA to preserve flexibility in how sponsors demonstrate scientific rigor so that third-party participation does not become a de facto requirement, create …”National Health Council The full filing →Industry · Holds data and adjudicates
“Yes, to maintain sequestered datasets and to serve as independent adjudicators. An independent party should have no financial interest in the sponsor or the foundation model developer.”Veritas Biomedical Analytics, Inc. The full filing →Clinicians · Backs third-party evaluation
“ACR supports the use of qualified, independent third parties for benchmarking, clinical confirmation, and postmarket assessment.”American College of Radiology (ACR) The full filing →
“Favor independent adjudication for high-stakes endpoints while avoiding a universal third-party testing barrier.”An individual (unknown) The full filing →Other · Voluntary, not mandatory
“We also support a voluntary role for qualified independent organizations, including accreditation and certification bodies, as long as FDA recognizes more than one pathway and …”ATA Action The full filing →Industry · Holds data and adjudicates
“Yes, to maintain sequestered datasets and to serve as independent adjudicators. An independent party should have no financial interest in the sponsor or the foundation model developer.”Veritas Biomedical Analytics, Inc. The full filing →
Only 15 filers in this group, so counts are shown, not shares.
Only 1 filer in this group, so counts are shown, not shares.
Of the 74 who gave a verdict, 65 accepted with changes, 3 accepted as written, 3 were split and 3 rejected.
Of the 44 who gave a verdict, 35 accepted with changes, 3 accepted as written, 3 were split and 3 rejected.
All 19 who gave a verdict accepted with changes.
Both who gave a verdict accepted with changes.
No one in this group gave a yes or no.
All 9 who gave a verdict accepted with changes.
Only 15 filers in this group, so counts are shown, not shares.
Only 1 filer in this group, so counts are shown, not shares.
CDRH is exploring whether a competency-based approach could be applied to devices that incorporate a variety of underlying model architectures, such as multimodal vision-language models and generative or predictive world models. Are there aspects of the described approach that might be ineffective or inapplicable to these devices? What additional or distinct considerations might need to be addressed?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 113 specific asks linked to Q17 (more oversight 59, mixed 2, less oversight 2, neither: method or clarity 50). Proposals: of the 85 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Regulate the clinical capability. Let the architecture change underneath it.”Ravi Pankhaniya, MD The full filing →Industry · Draws a sharp architecture line
“CDRH’s framework should draw a sharp architectural and evidentiary boundary between discriminative pixel-recognition models (CNNs) and generative Vision-Language Models (VLMs)”DeepHealth The full filing →Other · Wants modality-specific extensions
“We recommend that CDRH treat the existing elements as a core set and publish architecture- or modality-speci c extensions, rather than assuming the text-oriented elements apply …”An individual (unknown) The full filing →
“The competency-based structure carries over to multimodal and world-model devices, but the benchmark content must match their functions.”Sia Partners The full filing →Industry · Architecture-neutral approach
“We support a competency-based approach that is architecture-neutral.”Amazon.com, Inc. The full filing →Industry · Regulate capability, not architecture
“Regulate the clinical capability. Let the architecture change underneath it.”Ravi Pankhaniya, MD The full filing →
“The same competency architecture can apply to vision-language and predictive world models, but the task tests must change.”American Association of Physicists in Medicine The full filing →Clinicians · Needs perceptual, temporal tests
“Multimodal vision- language and predictive models require perceptual, temporal, and closed-loop testing that knowledge and dialogue benchmarks do not provide.”American Society for Gastrointestinal Endoscopy The full filing →Industry · Regulate capability, not architecture
“Regulate the clinical capability. Let the architecture change underneath it.”Ravi Pankhaniya, MD The full filing →
“Traditional benchmarking methods may not fully capture the performance of multimodal GenAI-enabled devices because these systems must interpret and integrate multiple sources of …”Loma Linda University School of Medicine (Harine Park) The full filing →Researchers · Link text, image tests
“I recommend making the relationship between these tests explicit for devices that combine text with images or other patient data.”Binesh Sadanandan The full filing →Industry · Regulate capability, not architecture
“Regulate the clinical capability. Let the architecture change underneath it.”Ravi Pankhaniya, MD The full filing →
“The NHC encourages FDA to preserve a technology- and architecture-neutral regulatory framework focused on intended use, patient risk, and demonstrated performance rather than …”National Health Council The full filing →Industry · Regulate capability, not architecture
“Regulate the clinical capability. Let the architecture change underneath it.”Ravi Pankhaniya, MD The full filing →Industry · Draws a sharp architecture line
“CDRH’s framework should draw a sharp architectural and evidentiary boundary between discriminative pixel-recognition models (CNNs) and generative Vision-Language Models (VLMs)”DeepHealth The full filing →
“FDA could apply a competency-based approach across different model architectures, provided the competencies, tests, and acceptance criteria reflect what the device is intended to do.”Scarlet The full filing →Other · Wants modality-specific extensions
“We recommend that CDRH treat the existing elements as a core set and publish architecture- or modality-speci c extensions, rather than assuming the text-oriented elements apply …”An individual (unknown) The full filing →Industry · Regulate capability, not architecture
“Regulate the clinical capability. Let the architecture change underneath it.”Ravi Pankhaniya, MD The full filing →
Only 16 filers in this group, so counts are shown, not shares.
Only 6 filers in this group, so counts are shown, not shares.
Only 1 filer in this group, so counts are shown, not shares.
Only 14 filers in this group, so counts are shown, not shares.
Only 13 asks in this group, so counts are shown, not shares.
Only 12 asks in this group, so counts are shown, not shares.
Only 2 asks in this group, so counts are shown, not shares.
CDRH is considering whether it may be appropriate to accept greater premarket uncertainty regarding a GenAI-enabled device’s benefit-risk profile through greater reliance on postmarket monitoring. Under what conditions might such an approach be appropriate, and what characteristics of a monitoring program would need to be in place to justify reduced premarket evidence? Are there device types or risk profiles for which this approach would not be appropriate?
In the summary report: Takeaway 6, Closer monitoring in place of some premarket testing got a conditional yes
Answer chart: of the 243 filers who answered (yes, with conditions 187, yes, broadly 3, no, it only verifies 15, no side 38). Proposals: of the 243 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Strong postmarket monitoring may justify some premarket uncertainty when actions are reversible, monitoring is timely and representative, harm can be detected before it spreads, …”ARPA-H ADVOCATE Program; Applied Physics Lab at Johns Hopkins University The full filing →Industry · Lower-risk uses only
“FDA should accept less evidence before authorization only for lower-risk uses when errors are reversible and dependable human review remains in place.”OCHIN The full filing →Clinicians · Rejects the trade-off
“The CAP firmly believes that greater reliance on postmarket monitoring is not an acceptable trade-off for greater premarket uncertainty.”College of American Pathologists The full filing →
“Align supports greater reliance on postmarket monitoring for lower-consequence and HCP-supervised functions, conditional on a prespecified monitoring plan with defined metrics, …”Align Technology Inc. The full filing →Industry · Lower-risk uses only
“FDA should accept less evidence before authorization only for lower-risk uses when errors are reversible and dependable human review remains in place.”OCHIN The full filing →Other · Yes if reversible, detectable
“Strong postmarket monitoring may justify some premarket uncertainty when actions are reversible, monitoring is timely and representative, harm can be detected before it spreads, …”ARPA-H ADVOCATE Program; Applied Physics Lab at Johns Hopkins University The full filing →
“Greater premarket uncertainty may be acceptable only for lower-risk, reviewable functions when the expected benefits are meaningful and a mature, enforceable monitoring …”Radiological Society of North America The full filing →Clinicians · Rejects the trade-off
“The CAP firmly believes that greater reliance on postmarket monitoring is not an acceptable trade-off for greater premarket uncertainty.”College of American Pathologists The full filing →Other · Yes if reversible, detectable
“Strong postmarket monitoring may justify some premarket uncertainty when actions are reversible, monitoring is timely and representative, harm can be detected before it spreads, …”ARPA-H ADVOCATE Program; Applied Physics Lab at Johns Hopkins University The full filing →
“Trading premarket evidence for postmarket monitoring is defensible only for failure modes the monitoring can actually detect.”David Liebovitz The full filing →Researchers · Tests before promises
“The FDA should not allow sponsors to replace required safety testing before market authorization with promises to study a GenAI-enabled device only after the device is introduced …”Program On Regulation, Therapeutics, And Law (PORTAL) and MIT Critical Data The full filing →Other · Yes if reversible, detectable
“Strong postmarket monitoring may justify some premarket uncertainty when actions are reversible, monitoring is timely and representative, harm can be detected before it spreads, …”ARPA-H ADVOCATE Program; Applied Physics Lab at Johns Hopkins University The full filing →
“We ask FDA to strengthen postmarket monitoring of devices enabled by AI but without reducing standards for premarket evidence.”Generation Patient The full filing →Patients · Opposes replacing premarket evidence
“We disagree with replacing premarket evidence with postmarket monitoring, or weighing it heavier than premarket evidence, because it would result in continued harm to patients.”Deinfluence; Doctors for America; Generation Patient; InnovateEDU; Innovation For Everyone; Interfaith Center on Corporate Responsibility (ICCR); National Center for Health Research (NCHR); Our Bodies Ourselves; Patient Safety Action Network; USA Patient Network; Washington Advocates for Patient Safety; Young People’s Alliance The full filing →Other · Yes if reversible, detectable
“Strong postmarket monitoring may justify some premarket uncertainty when actions are reversible, monitoring is timely and representative, harm can be detected before it spreads, …”ARPA-H ADVOCATE Program; Applied Physics Lab at Johns Hopkins University The full filing →
“Strong postmarket monitoring may justify some premarket uncertainty when actions are reversible, monitoring is timely and representative, harm can be detected before it spreads, …”ARPA-H ADVOCATE Program; Applied Physics Lab at Johns Hopkins University The full filing →Other · No automatic evidence reduction
“Greater reliance on postmarket monitoring may be appropriate in some circumstances but should not automatically justify reduced premarket evidence.”HL7 Caliper Accelerator The full filing →Industry · Lower-risk uses only
“FDA should accept less evidence before authorization only for lower-risk uses when errors are reversible and dependable human review remains in place.”OCHIN The full filing →
Only 4 filers in this group, so counts are shown, not shares.
Only 4 filers in this group, so counts are shown, not shares.
Please comment on the potential approaches to postmarket performance evaluation, including periodic re-benchmarking, sample-based clinician review, and performance degradation monitoring. What additional approaches should CDRH consider, and how should the cadence and triggering events for reassessment be determined?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 678 specific asks linked to Q19 (more oversight 472, mixed 11, less oversight 18, neither: method or clarity 177). Proposals: of the 357 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Treat every upstream model change as a re-benchmarking trigger.”Elgaraali Rodwan The full filing →Other · Cadence plus event triggers
“Reassessment should combine a defined cadence with event-based triggers.”An individual (unknown) The full filing →Industry · Sponsors set own thresholds
“Since sponsors are the most knowledgeable about their devices, they should be permitted to specify the analyses and thresholds used to detect performance degradation.”Baxter The full filing →
“The three approaches are complementary because each catches failures the others miss.”Sia Partners The full filing →Industry · Sponsors set own thresholds
“Since sponsors are the most knowledgeable about their devices, they should be permitted to specify the analyses and thresholds used to detect performance degradation.”Baxter The full filing →Clinicians · Re-test on model change
“Treat every upstream model change as a re-benchmarking trigger.”Elgaraali Rodwan The full filing →
“Treat every upstream model change as a re-benchmarking trigger.”Elgaraali Rodwan The full filing →Clinicians · Track near misses
“Near misses matter. Workarounds matter. Repeated concerns that never become adverse events matter.”Kimberly Maurer The full filing →Other · Cadence plus event triggers
“Reassessment should combine a defined cadence with event-based triggers.”An individual (unknown) The full filing →
“I support periodic re-benchmarking, clinician review, and degradation monitoring, but consider them complementary rather than interchangeable approaches.”Martin Haimerl The full filing →Researchers · Encourages re-benchmarking
“Re-benchmarking should be encouraged to determine if capabilities have changed after the premarket evaluation.”Loma Linda University School of Medicine (Harine Park) The full filing →Clinicians · Re-test on model change
“Treat every upstream model change as a re-benchmarking trigger.”Elgaraali Rodwan The full filing →
“The FDA should mandate that human medical practitioners are employed to regularly review chat logs to evaluate response accuracy and determine whether further escalation triggers …”Tech Justice Law The full filing →Patients · One-time clearance inadequate
“A traditional, one-time clearance is completely inadequate for stochastic, evolving models that can drift or change behavior post-deployment.”Westervelt Kim The full filing →Clinicians · Re-test on model change
“Treat every upstream model change as a re-benchmarking trigger.”Elgaraali Rodwan The full filing →
“Reassessment should combine a defined cadence with event-based triggers.”An individual (unknown) The full filing →Other · Complementary layers, not alternatives
“The three approaches the paper lists answer different questions and should be treated as complementary layers, not alternatives.”An individual (unknown) The full filing →Clinicians · Re-test on model change
“Treat every upstream model change as a re-benchmarking trigger.”Elgaraali Rodwan The full filing →
Only 6 filers in this group, so counts are shown, not shares.
Only 17 asks in this group, so counts are shown, not shares.
Please comment on whether the proposed approaches to postmarket monitoring can be facilitated by machine-based supervisory agents. What considerations, including the evaluation and reliability of the supervisory agent itself, should CDRH take into account for such an approach?
In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes
Answer chart: of the 201 filers who answered (accept, with changes 67, accept, as written 2, split 1, reject 0, no verdict 131). Proposals: of the 201 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“It can usefully triage and prioritize cases for human review; it should not replace sample-based human review, and it should itself be validated independently, with disclosed …”Navid Farr The full filing →Industry · Only approach that scales
“Machine-based supervisory agents are feasible, and for GenAI-enabled devices with high interaction volumes and open-ended outputs, they are likely the only approach that scales”Sentir Health The full filing →Industry · Not assurance until validated
“No independent basis has been offered for placing greater confidence in the supervisor than in the supervised function. …”Lukas Block The full filing →
“One AI system may help monitor another, but it should not be the only safeguard.”OCHIN The full filing →Industry · Qualify the supervisor first
“A supervisory agent may support postmarket monitoring, but it should be independently qualified and validated for that role.”BioPhorum The full filing →Industry · Triage aid, validated first
“It can usefully triage and prioritize cases for human review; it should not replace sample-based human review, and it should itself be validated independently, with disclosed …”Navid Farr The full filing →
“Machine-based supervisory agents may support monitoring at scale, but they should triage cases for human review rather than serve as the sole judge of safety for higher-risk devices.”American Society for Gastrointestinal Endoscopy The full filing →Clinicians · AI not final judge
“Yes. But AI cannot be permitted to become the final judge of its own safety.”Deborah Ault, RN The full filing →Industry · Triage aid, validated first
“It can usefully triage and prioritize cases for human review; it should not replace sample-based human review, and it should itself be validated independently, with disclosed …”Navid Farr The full filing →
“Automated monitoring systems should be validated before their results are relied on (Q20).”NSF AI Research Institute on Interaction for AI Assistants (ARIA) The full filing →Researchers · Flag only, humans act
“Machine-based screening tools may be needed for comprehensive postmarket monitoring, and, when appropriate, they should flag issues for human review rather than act on their own.”Stanford Institute for Human-Centered Artificial Intelligence (HAI) The full filing →Industry · Triage aid, validated first
“It can usefully triage and prioritize cases for human review; it should not replace sample-based human review, and it should itself be validated independently, with disclosed …”Navid Farr The full filing →
“Their use, however, does not eliminate the need to evaluate the monitoring function itself.”National Health Council The full filing →Industry · Triage aid, validated first
“It can usefully triage and prioritize cases for human review; it should not replace sample-based human review, and it should itself be validated independently, with disclosed …”Navid Farr The full filing →Industry · Only approach that scales
“Machine-based supervisory agents are feasible, and for GenAI-enabled devices with high interaction volumes and open-ended outputs, they are likely the only approach that scales”Sentir Health The full filing →
“Use supervisory agents to screen and prioritize, not as the only safety control.”ARPA-H ADVOCATE Program; Applied Physics Lab at Johns Hopkins University The full filing →Other · Test for shared failures
“Supervisory technologies require evaluation of their own reliability, including whether they share failure modes with the systems they monitor.”Main Street Foundation Center for Regulatory Analysis and Engagement; Main Street Foundation The full filing →Industry · Triage aid, validated first
“It can usefully triage and prioritize cases for human review; it should not replace sample-based human review, and it should itself be validated independently, with disclosed …”Navid Farr The full filing →
Only 15 filers in this group, so counts are shown, not shares.
Only 1 filer in this group, so counts are shown, not shares.
Of the 70 who gave a verdict, 67 accepted with changes, 2 accepted as written and 1 was split.
Of the 50 who gave a verdict, 48 accepted with changes, 1 accepted as written and 1 was split.
All 11 who gave a verdict accepted with changes.
Both who gave a verdict accepted with changes.
No one in this group gave a yes or no.
Of the 7 who gave a verdict, 6 accepted with changes and 1 accepted as written.
Only 15 filers in this group, so counts are shown, not shares.
Only 1 filer in this group, so counts are shown, not shares.
What roles might clinicians, healthcare institutions, professional societies, standards-setting bodies, and other stakeholders appropriately play in postmarket monitoring, and how can these roles be structured without diffusing manufacturer accountability?
In the summary report: Takeaway 7, The device maker stays responsible, a majority said, even for AI it did not build
Answer chart: of the 208 filers who took a side on this point (manufacturer first 180, whoever controls the change 28). The answer chart counts only filers who named one option; proposal shares are of all who answered. Proposals: of the 251 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Primary responsibility for monitoring must remain with the manufacturer because it controls the system’s design and updates, has access to performance data across deployments, and …”American Psychiatric Association The full filing →Industry · Wants shared accountability
“The post-market monitoring framework should be one of shared accountability among all stakeholders.”Wolters Kluwer The full filing →Clinicians · No burden on nurses
“Manufacturers must keep accountability; hospitals and frontline nurses should not inherit an undefined surveillance burden.”Robert Elvin Domondon The full filing →
“The accountability principle can be stated simply, and I recommend CDRH state it: information may come from anywhere; the duty to investigate and act remains with the manufacturer.”Hari Prakash Chanumolu The full filing →Industry · Keeps duties off hospitals
“keep accountability with the manufacturer and not transfer monitoring obligations to deploying institutions, so that health systems are not deterred from adopting these devices”AI Healthcare Coalition The full filing →Clinicians · Manufacturer keeps monitoring duty
“Primary responsibility for monitoring must remain with the manufacturer because it controls the system’s design and updates, has access to performance data across deployments, and …”American Psychiatric Association The full filing →
“Primary responsibility for monitoring must remain with the manufacturer because it controls the system’s design and updates, has access to performance data across deployments, and …”American Psychiatric Association The full filing →Clinicians · No burden on nurses
“Manufacturers must keep accountability; hospitals and frontline nurses should not inherit an undefined surveillance burden.”Robert Elvin Domondon The full filing →Industry · Wants shared accountability
“The post-market monitoring framework should be one of shared accountability among all stakeholders.”Wolters Kluwer The full filing →
“On Question 21: a commitment registry allocates roles without diffusing accountability. The sponsor declares its configuration and remains accountable for the device.”Tomonoshin Aoki The full filing →Researchers · Not on hospitals alone
“However, the responsibility of postmarket monitoring cannot rest on the healthcare institutions adopting the tools alone.”Stanford Institute for Human-Centered Artificial Intelligence (HAI) The full filing →Clinicians · Manufacturer keeps monitoring duty
“Primary responsibility for monitoring must remain with the manufacturer because it controls the system’s design and updates, has access to performance data across deployments, and …”American Psychiatric Association The full filing →
“Patient organizations can also help identify patterns across individual experiences and advise whether emerging issues affect outcomes important to their communities.”The ALS Association The full filing →Clinicians · Manufacturer keeps monitoring duty
“Primary responsibility for monitoring must remain with the manufacturer because it controls the system’s design and updates, has access to performance data across deployments, and …”American Psychiatric Association The full filing →Industry · Wants shared accountability
“The post-market monitoring framework should be one of shared accountability among all stakeholders.”Wolters Kluwer The full filing →
“The manufacturer should remain accountable for postmarket monitoring, with other stakeholders contributing evidence to its processes rather than sharing responsibility for them.”Scarlet The full filing →Other · Wants low-barrier reporting
“PSO-style incident reporting offers clinicians a low-barrier channel to flag subtle failures that might not meet the threshold for formal reporting.”Milan Toma The full filing →Clinicians · Manufacturer keeps monitoring duty
“Primary responsibility for monitoring must remain with the manufacturer because it controls the system’s design and updates, has access to performance data across deployments, and …”American Psychiatric Association The full filing →
Only 2 filers in this group, so counts are shown, not shares.
Only 15 filers who took a side in this group, so counts are shown, not shares.
Only 2 filers who took a side in this group, so counts are shown, not shares.
CDRH envisions that a manufacturer’s premarket competency-based assessment might serve as a baseline against which post-deployment modifications could be re-evaluated. How might the extent of re-benchmarking or other evidence be scaled to the nature and expected impact of a given modification? For example, are there categories of change that might not significantly affect safety or effectiveness of a GenAI-enabled device, or might be appropriately managed within a sponsor’s quality management system versus requiring FDA premarket review and authorization before implementation? Are there categories of changes that might be appropriate for inclusion in a PCCP?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 439 specific asks linked to Q22 (more oversight 170, mixed 24, less oversight 53, neither: method or clarity 192). Proposals: of the 231 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Mayo Clinic recommends that FDA scale re-evaluation requirements according to a modification’s plausible effect on clinical behavior and the evidence supporting the validated …”Mayo Clinic The full filing →Patients · New model needs submission
“FDA should also treat a model substitution made outside an approved change control plan as a modification requiring a new premarket submission.”Generation Patient The full filing →Other · Routine changes skip review
“Most routine iteration within the authorized intended use shouldn’t require new premarket review.”Curai Health The full filing →
“The amount of reassessment should reflect the potential clinical effect of the change, regardless of how small the change appears in the code.”Clara Health The full filing →Industry · New model, full re-test
“A change to the foundation model, the intended use or the way outputs are presented should trigger full re-benchmarking. …”OLIN The full filing →Clinicians · Scale to clinical effect
“Mayo Clinic recommends that FDA scale re-evaluation requirements according to a modification’s plausible effect on clinical behavior and the evidence supporting the validated …”Mayo Clinic The full filing →
“Reassessment after a modification should depend on the expected clinical effect of the change rather than the size of the software change.”Johns Hopkins Medicine The full filing →Clinicians · Re-test matches the change
“The size of the re-test should match the size of the change. A configuration change needs a targeted check. A new underlying model needs the full baseline again.”Binita S. Ashar The full filing →Clinicians · Scale to clinical effect
“Mayo Clinic recommends that FDA scale re-evaluation requirements according to a modification’s plausible effect on clinical behavior and the evidence supporting the validated …”Mayo Clinic The full filing →
“The extent of re-benchmarking should follow the modification’s plausible clinical effects: use targeted comparisons when its impact is well bounded and broader testing when it is …”People’s Evidence Lab Inc. The full filing →Researchers · Re-test affected behavior
“Post-update testing should therefore cover the affected clinical behavior, even when the model architecture is unchanged.”Binesh Sadanandan The full filing →Clinicians · Scale to clinical effect
“Mayo Clinic recommends that FDA scale re-evaluation requirements according to a modification’s plausible effect on clinical behavior and the evidence supporting the validated …”Mayo Clinic The full filing →
“FDA should also treat a model substitution made outside an approved change control plan as a modification requiring a new premarket submission.”Generation Patient The full filing →Patients · Wants objective change criteria
“FDA should encourage sponsors to define objective criteria for determining when a change can be managed within the quality system, when PCCP testing is appropriate, and when a new …”National Health Council The full filing →Clinicians · Scale to clinical effect
“Mayo Clinic recommends that FDA scale re-evaluation requirements according to a modification’s plausible effect on clinical behavior and the evidence supporting the validated …”Mayo Clinic The full filing →
“Most routine iteration within the authorized intended use shouldn’t require new premarket review.”Curai Health The full filing →Other · Separate routine from material
“FDA should distinguish routine maintenance from modifications that materially affect clinical behavior, intended use, or patient risk.”Main Street Foundation Center for Regulatory Analysis and Engagement; Main Street Foundation The full filing →Clinicians · Scale to clinical effect
“Mayo Clinic recommends that FDA scale re-evaluation requirements according to a modification’s plausible effect on clinical behavior and the evidence supporting the validated …”Mayo Clinic The full filing →
Only 19 filers in this group, so counts are shown, not shares.
Only 3 filers in this group, so counts are shown, not shares.
Only 3 asks in this group, so counts are shown, not shares.
How might PCCP concepts or other change-control approaches be adapted for GenAI-enabled devices when the nature or scope of future modifications cannot be fully prespecified?
In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed
Asks chart: of the 235 specific asks linked to Q23 (more oversight 71, mixed 24, less oversight 48, neither: method or clarity 92). Proposals: of the 176 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“When future changes cannot be fully prespecified, we recommend a simple principle: specify the test, not the change.”OLIN The full filing →Clinicians · No PCCP for unknowns
“Fundamentally, if a sponsor cannot predetermine the change, including specifying in advance the modification, validation protocols, and acceptance criteria, a PCCP is not the …”American College of Radiology (ACR) The full filing →Patients · Wants guardrails on categories
“The NHC encourages FDA to provide sufficient guardrails around broadly defined change categories so that flexibility does not reduce clarity regarding when additional testing or …”National Health Council The full filing →
“When future changes cannot be fully prespecified, we recommend a simple principle: specify the test, not the change.”OLIN The full filing →Industry · Set testable change boundaries
“PCCPs should define testable change boundaries rather than attempt to enumerate every future modification.”Johnson & Johnson The full filing →Clinicians · No PCCP for unknowns
“Fundamentally, if a sponsor cannot predetermine the change, including specifying in advance the modification, validation protocols, and acceptance criteria, a PCCP is not the …”American College of Radiology (ACR) The full filing →
“Fundamentally, if a sponsor cannot predetermine the change, including specifying in advance the modification, validation protocols, and acceptance criteria, a PCCP is not the …”American College of Radiology (ACR) The full filing →Clinicians · No open-ended permission
“Prespecifying a validation process is useful, but it should not provide open-ended permission for uncharacterized clinical behavior.”Keystone Neuroinformatics The full filing →Industry · Specify the test
“When future changes cannot be fully prespecified, we recommend a simple principle: specify the test, not the change.”OLIN The full filing →
“PCCP concepts can be adapted by specifying how the changes will be assessed, even when the exact changes cannot be predicted.”Loma Linda University School of Medicine (Harine Park) The full filing →Researchers · Plan sets review point
“A predetermined change control plan should state the kinds of change permitted, their limits, the testing each requires, and the point at which the FDA must review the device again.”Program On Regulation, Therapeutics, And Law (PORTAL) and MIT Critical Data The full filing →Industry · Specify the test
“When future changes cannot be fully prespecified, we recommend a simple principle: specify the test, not the change.”OLIN The full filing →
“The NHC encourages FDA to provide sufficient guardrails around broadly defined change categories so that flexibility does not reduce clarity regarding when additional testing or …”National Health Council The full filing →Industry · Specify the test
“When future changes cannot be fully prespecified, we recommend a simple principle: specify the test, not the change.”OLIN The full filing →Clinicians · No PCCP for unknowns
“Fundamentally, if a sponsor cannot predetermine the change, including specifying in advance the modification, validation protocols, and acceptance criteria, a PCCP is not the …”American College of Radiology (ACR) The full filing →
“For GenAI-enabled devices, a PCCP that lists each specific future modification will often be impractical.”Connected Health Initiative The full filing →Other · Not a blanket authorization
“FDA’s August 2025 final PCCP guidance already describes marketing-submission recommendations for specified AI-device modifications, but it is not a blanket authorization for …”An individual (unknown) The full filing →Industry · Specify the test
“When future changes cannot be fully prespecified, we recommend a simple principle: specify the test, not the change.”OLIN The full filing →
Only 11 filers in this group, so counts are shown, not shares.
Only 1 filer in this group, so counts are shown, not shares.
Only 10 asks in this group, so counts are shown, not shares.
Only 1 asks in this group, so counts are shown, not shares.
For devices built on third-party foundation models, changes to the underlying model may be initiated by the third-party model developer rather than the device manufacturer. How can a manufacturer detect, evaluate, and respond to such changes in a timely manner? What mechanisms—for example, contractual, technical, or through a PCCP—could provide reasonable assurance that third-party developer-initiated changes do not compromise the safety or effectiveness of the device?
In the summary report: Takeaway 7, The device maker stays responsible, a majority said, even for AI it did not build
Asks chart: of the 559 specific asks linked to Q24 (more oversight 343, mixed 10, less oversight 30, neither: method or clarity 176). Proposals: of the 260 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“In response to the discussion paper’s question on third-party foundation model changes (Q24), we recommend that CDRH treat any change to the underlying foundation model as a …”Alterra Labs The full filing →Researchers · Maker controls model updates
“Manufacturers should be in control when model updates reach patients, instead of accepting changes automatically from any third-party model developer.”Loma Linda University School of Medicine (Harine Park) The full filing →Other · Small makers lack leverage
“In practice, many manufacturers, particularly smaller ones, have limited leverage over major model providers, so their technical measures and planned fallbacks should not depend …”Scarlet The full filing →
“Useful mechanisms include pinned model versions with no silent updates, contractual advance notice of changes, and re-benchmarking before a changed model is deployed.”Aidoc The full filing →Industry · Small makers lack leverage
“A small manufacturer cannot compel a foundation model developer to disclose changes or give advance notice, and contractual leverage is concentrated in the largest sponsors.”Richard Pescatore, DO (BellyMD) The full filing →Industry · Re-test before patients
“In response to the discussion paper’s question on third-party foundation model changes (Q24), we recommend that CDRH treat any change to the underlying foundation model as a …”Alterra Labs The full filing →
“Manufacturers should run a pinned model version and re-benchmark before adopting a vendor update, supported by contractual notice obligations.”Salvo Health The full filing →Clinicians · Require contract rights
“FDA should require, as a condition of authorization, enforceable contractual rights to advance notice of material model changes, version pinning, configuration logs, incident …”Doctors for America The full filing →Industry · Re-test before patients
“In response to the discussion paper’s question on third-party foundation model changes (Q24), we recommend that CDRH treat any change to the underlying foundation model as a …”Alterra Labs The full filing →
“Manufacturers should be in control when model updates reach patients, instead of accepting changes automatically from any third-party model developer.”Loma Linda University School of Medicine (Harine Park) The full filing →Researchers · Notice, pinning, sentinel tests
“Use contractual notice, version pinning where available, request-time checks against provider records, and sentinel testing for unexpected behavioral change.”OHDSI Medical Devices Working Group (Junhwan Kwon) The full filing →Industry · Re-test before patients
“In response to the discussion paper’s question on third-party foundation model changes (Q24), we recommend that CDRH treat any change to the underlying foundation model as a …”Alterra Labs The full filing →
“FDA should require device sponsors to identify each model and version their product incorporates, including fallbacks or alternates.”Generation Patient The full filing →Industry · Re-test before patients
“In response to the discussion paper’s question on third-party foundation model changes (Q24), we recommend that CDRH treat any change to the underlying foundation model as a …”Alterra Labs The full filing →Researchers · Maker controls model updates
“Manufacturers should be in control when model updates reach patients, instead of accepting changes automatically from any third-party model developer.”Loma Linda University School of Medicine (Harine Park) The full filing →
“In practice, many manufacturers, particularly smaller ones, have limited leverage over major model providers, so their technical measures and planned fallbacks should not depend …”Scarlet The full filing →Other · Pin model snapshots
“Manufacturers should pin each device to a specific model snapshot so it can’t be updated automatically.”Curai Health The full filing →Industry · Re-test before patients
“In response to the discussion paper’s question on third-party foundation model changes (Q24), we recommend that CDRH treat any change to the underlying foundation model as a …”Alterra Labs The full filing →
Only 3 filers in this group, so counts are shown, not shares.
Only 6 asks in this group, so counts are shown, not shares.
Would voluntary Foundation Model MAFs be practical for foundation model developers to provide and to keep current, and what content might they include? Given that participation would be voluntary and that model developers may have limited incentive to disclose safety-relevant information, what would make such a program sufficiently useful for premarket review? Are there alternative mechanisms CDRH should consider for obtaining information about underlying foundation models?
In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes
Answer chart: of the 179 filers who answered (accept, with changes 85, accept, as written 12, split 0, reject 2, no verdict 80). Proposals: of the 179 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Voluntary Foundation Model Master Files may be useful if they contain standardized, current, and sufficiently detailed information for FDA and device manufacturers to assess …”Danaher Corporation The full filing →Other · Keep files voluntary
“CHI supports FDA’s proposal to keep Foundation Model Master Files (MAFs) voluntary and its statement that holding a MAF does not authorize a model for any intended use.”Connected Health Initiative The full filing →Industry · Wants required disclosure
“No, these programs would not work because foundation model developers have no incentive to participate. …”TechInHSR LLC (Tamiko Eto) The full filing →
“Voluntary Foundation Model Master Files may be useful if they contain standardized, current, and sufficiently detailed information for FDA and device manufacturers to assess …”Danaher Corporation The full filing →Industry · Helps small manufacturers
“We support voluntary Foundation Model MAFs. Their main value is to small manufacturers.”Grasshopper Health, Inc. The full filing →Other · Keep files voluntary
“CHI supports FDA’s proposal to keep Foundation Model Master Files (MAFs) voluntary and its statement that holding a MAF does not authorize a model for any intended use.”Connected Health Initiative The full filing →
“A Foundation Model Device Master File (MAF) is a useful idea, but a voluntary, confidential file cannot carry the safety weight these models bear.”Doctors for America The full filing →Clinicians · Transparency to end users
“Require transparency to end users, not only to FDA”National Nurse Practitioner Entrepreneur Network The full filing →Industry · Useful if standardized, current
“Voluntary Foundation Model Master Files may be useful if they contain standardized, current, and sufficiently detailed information for FDA and device manufacturers to assess …”Danaher Corporation The full filing →
“Voluntary Foundation Model Master Files would support review, but voluntary disclosure alone is not enough.”NSF AI Research Institute on Interaction for AI Assistants (ARIA) The full filing →Researchers · Practical with a template
“Voluntary MAFs are practical if a structured template is provided.”Javid Rahaman The full filing →Industry · Useful if standardized, current
“Voluntary Foundation Model Master Files may be useful if they contain standardized, current, and sufficiently detailed information for FDA and device manufacturers to assess …”Danaher Corporation The full filing →
“The NHC supports FDA’s exploration of voluntary Foundation Model Device Master Files (MAFs).”National Health Council The full filing →Patients · Voluntary file is toothless
“Yet CDRH concedes that model developers "may have limited incentive to disclose safety-relevant information," 11 which would make a voluntary mechanism toothless.”Deinfluence; Doctors for America; Generation Patient; InnovateEDU; Innovation For Everyone; Interfaith Center on Corporate Responsibility (ICCR); National Center for Health Research (NCHR); Our Bodies Ourselves; Patient Safety Action Network; USA Patient Network; Washington Advocates for Patient Safety; Young People’s Alliance The full filing →Industry · Useful if standardized, current
“Voluntary Foundation Model Master Files may be useful if they contain standardized, current, and sufficiently detailed information for FDA and device manufacturers to assess …”Danaher Corporation The full filing →
“A voluntary program is useful, but cannot be the only mechanism.”Associazione Tendenze Salute e Sanità The full filing →Other · Keep files voluntary
“CHI supports FDA’s proposal to keep Foundation Model Master Files (MAFs) voluntary and its statement that holding a MAF does not authorize a model for any intended use.”Connected Health Initiative The full filing →Industry · Useful if standardized, current
“Voluntary Foundation Model Master Files may be useful if they contain standardized, current, and sufficiently detailed information for FDA and device manufacturers to assess …”Danaher Corporation The full filing →
Only 14 filers in this group, so counts are shown, not shares.
Only 3 filers in this group, so counts are shown, not shares.
Of the 99 who gave a verdict, 85 accepted with changes, 12 accepted as written and 2 rejected.
Of the 64 who gave a verdict, 53 accepted with changes, 9 accepted as written and 2 rejected.
Of the 14 who gave a verdict, 13 accepted with changes and 1 accepted as written.
All 7 who gave a verdict accepted with changes.
The one filer who gave a verdict accepted with changes.
Of the 13 who gave a verdict, 11 accepted with changes and 2 accepted as written.
Only 14 filers in this group, so counts are shown, not shares.
Only 3 filers in this group, so counts are shown, not shares.
Are there additional considerations that inform the premarket and postmarket evaluation of agentic GenAI-enabled devices, beyond those applicable to non-agentic GenAI-enabled devices? How should the elevated risk associated with autonomous multi-step action, tool use, and reduced opportunity for human review be reflected in acceptance criteria and oversight?
In the summary report: Takeaway 8, How much an AI may do on its own should depend on the risk, filers said
Asks chart: of the 508 specific asks linked to Q26 (more oversight 378, mixed 6, less oversight 11, neither: method or clarity 113). Proposals: of the 232 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted
“Agentic devices should also require human authorization before irreversible or high-consequence actions, support halt and rollback, and keep tamper-evident logs that allow the …”Edge Compliance The full filing →Clinicians · Clinician review before action
“For agentic systems, the FDA should also require clear points where a clinician must review the system's work before it takes an irreversible action.”Felipe Avila The full filing →Industry · No separate agentic framework
“Medical Alley does not recommend a separate regulatory framework solely because a GenAI-enabled device is agentic.”Medical Alley Artificial Intelligence Task Force; Medical Alley The full filing →
“Agentic devices should also require human authorization before irreversible or high-consequence actions, support halt and rollback, and keep tamper-evident logs that allow the …”Edge Compliance The full filing →Industry · No separate agentic framework
“Medical Alley does not recommend a separate regulatory framework solely because a GenAI-enabled device is agentic.”Medical Alley Artificial Intelligence Task Force; Medical Alley The full filing →Clinicians · Clinician review before action
“For agentic systems, the FDA should also require clear points where a clinician must review the system's work before it takes an irreversible action.”Felipe Avila The full filing →
“For agentic systems, the FDA should also require clear points where a clinician must review the system's work before it takes an irreversible action.”Felipe Avila The full filing →Clinicians · Heightened oversight needed
“Agentic devices require heightened oversight because errors can compound across multiple steps before a clinician can intervene.”American Medical Informatics Association The full filing →Industry · Human sign-off and logs
“Agentic devices should also require human authorization before irreversible or high-consequence actions, support halt and rollback, and keep tamper-evident logs that allow the …”Edge Compliance The full filing →
“Yes, agentic GenAI-enabled devices would need additional evaluation since they can create mistakes that affect future steps before a clinician can intervene.”Loma Linda University School of Medicine (Harine Park) The full filing →Researchers · Criteria set per action
“Acceptance criteria for an agentic device should be set against the floor for each action it can take, with the highest-consequence actions measured most carefully.”Rohith Reddy Bellibatlu (Independent Researcher, Clinical AI Evaluation Methodology) The full filing →Industry · Human sign-off and logs
“Agentic devices should also require human authorization before irreversible or high-consequence actions, support halt and rollback, and keep tamper-evident logs that allow the …”Edge Compliance The full filing →
“Agentic devices should also require human authorization before irreversible or high-consequence actions, support halt and rollback, and keep tamper-evident logs that allow the …”Edge Compliance The full filing →Clinicians · Clinician review before action
“For agentic systems, the FDA should also require clear points where a clinician must review the system's work before it takes an irreversible action.”Felipe Avila The full filing →Industry · No separate agentic framework
“Medical Alley does not recommend a separate regulatory framework solely because a GenAI-enabled device is agentic.”Medical Alley Artificial Intelligence Task Force; Medical Alley The full filing →
“For an agentic device, FDA should require a declared authority envelope identifying permitted tools, data, targets, actions, clinical contexts, and autonomy levels.”VivaSecuris The full filing →Other · Test whole action sequences
“Sponsors should evaluate agentic devices on the safety of complete action sequences, as well as the final clinical result.”Scarlet The full filing →Industry · Human sign-off and logs
“Agentic devices should also require human authorization before irreversible or high-consequence actions, support halt and rollback, and keep tamper-evident logs that allow the …”Edge Compliance The full filing →
Only 1 filer in this group, so counts are shown, not shares.
Only 3 asks in this group, so counts are shown, not shares.