MethodologyPrepared byRecovryAI

Results by question · Docket FDA-2026-N-7874 · Comments closed October 19, 2026 · Filings posted through November 3, 2026

The public response to each of the FDA’s 26 questions

For every question on generative AI medical devices: what 509 filers proposed, how they answered, and where kinds of filer differed.

509filers
26questions
Industry 45%Clinicians 24%Researchers 8%Patients 2%Other 20%

Answers come from the 504 filers read for content; their groups, by share

The 26 questions

Question 1 of 26 · The two-axis risk framework316 filers answered · 63% of all

Does a two-axis framework, AI device activity and the consequence of relying on an incorrect output, capture the dimensions of risk?

The FDA’s full question

Does the two-axis risk framework, organized around device activity and the consequence of relying on an incorrect output, appropriately capture the dimensions most relevant to the risk of a GenAI-enabled software function? If there are additional dimensions—such as the reversibility of a resulting action, the availability of downstream safeguards, the time pressure of the deployment setting, or the traceability of the output (i.e., to primary source materials)—that should be represented in a risk framework, please describe and provide examples of how they should be represented.

In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes · Takeaway 8, How much an AI may do on its own should depend on the risk, filers said

Key findingsFindings describe all filers.

  1. The two axes are accepted as a starting point.Nearly all who accept them still ask for changes.
  2. Filers add factors the FDA itself named.Reversibility, safeguards, sources and time pressure lead.
  3. Clinicians put more weight on time pressure.Named as a factor by 42% of clinicians and 26% of industry.
How these are counted

Answer chart: of the 316 filers who answered (accept, with changes 181, accept, as written 9, split 1, reject 3, no verdict 122). Proposals: of the 316 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q1 →

Do the two axes capture the risk?

57%39%
59%38%
68%30%

Only 19 filers in this group, so counts are shown, not shares.

811

Only 2 filers in this group, so counts are shown, not shares.

2
41%49%
Accept, with changesAccept, as writtenSplitRejectAnswered, no verdict

Of the 194 who gave a verdict, 181 accepted with changes, 9 accepted as written, 1 was split and 3 rejected.

Of the 107 who gave a verdict, 101 accepted with changes, 5 accepted as written and 1 rejected.

Of the 52 who gave a verdict, 50 accepted with changes, 1 accepted as written and 1 rejected.

All 8 who gave a verdict accepted with changes.

Both who gave a verdict accepted with changes.

Of the 25 who gave a verdict, 20 accepted with changes, 3 accepted as written, 1 was split and 1 rejected.

What filers proposed

FDA a factor the FDA itself suggested

  1. Reversibility of the actionFDA49%
  2. Downstream safeguardsFDA42%
  3. Tracing outputs to sourcesFDA34%
  4. Time pressureFDA32%
All 8 proposals
  1. Add as modifiers, not new axes23%
  2. Chance an error is caught21%
  3. Credit only proven safeguards19%
  4. Align with existing standards11%
  1. Reversibility of the actionFDA44%
  2. Downstream safeguardsFDA42%
  3. Tracing outputs to sourcesFDA38%
  4. Time pressureFDA26%
All 8 proposals
  1. Add as modifiers, not new axes22%
  2. Credit only proven safeguards18%
  3. Chance an error is caught17%
  4. Align with existing standards15%
  1. Reversibility of the actionFDA57%
  2. Downstream safeguardsFDA49%
  3. Time pressureFDA42%
  4. Credit only proven safeguards28%
All 8 proposals
  1. Tracing outputs to sourcesFDA26%
  2. Add as modifiers, not new axes23%
  3. Chance an error is caught23%
  4. Align with existing standards8%

Only 19 filers in this group, so counts are shown, not shares.

  1. Reversibility of the actionFDA9
  2. Add as modifiers, not new axes6
  3. Tracing outputs to sourcesFDA5
  4. Time pressureFDA5
All 8 proposals
  1. Chance an error is caught5
  2. Downstream safeguardsFDA3
  3. Credit only proven safeguards2
  4. Align with existing standards1

Only 2 filers in this group, so counts are shown, not shares.

  1. Reversibility of the actionFDA2
  2. Downstream safeguardsFDA2
  3. Chance an error is caught2
  4. Tracing outputs to sourcesFDA1
All 8 proposals
  1. Time pressureFDA1
  2. Add as modifiers, not new axes0
  3. Credit only proven safeguards0
  4. Align with existing standards0
  1. Reversibility of the actionFDA53%
  2. Downstream safeguardsFDA39%
  3. Time pressureFDA37%
  4. Tracing outputs to sourcesFDA31%
All 8 proposals
  1. Chance an error is caught27%
  2. Add as modifiers, not new axes24%
  3. Credit only proven safeguards12%
  4. Align with existing standards4%
Question 2 of 26 · The spectrum of device activity198 filers answered · 39% of all

How should the continuum from non-directive to action-directing outputs, and the risk that changes along it, be accounted for?

The FDA’s full question

CDRH seeks input on how to account for the spectrum of GenAI-enabled informational functions that vary in the degree to which they direct a user to a particular action (i.e., between “non-directive” and “action-directing”). What characteristics of an output—such as its wording, specificity, personalization, or context—could be considered as modifiers of the risk associated with an informational function, after accounting for the device’s overall functionality and intended use? What additional information would provide manufacturers with sufficient clarity and predictability around risk assessment for informational functions while recognizing that directiveness may exist along a continuum rather than as a binary distinction?

In the summary report: Takeaway 4, A disclaimer does not make AI advice safer, filers told the FDA

Key findingsFindings describe all filers.

  1. Effect, not wording, sets the risk.A consult-your-doctor disclaimer does not lower it, filers say.
  2. Filers want the FDA to name what raises risk.Such as how specific, urgent or certain an output sounds.
  3. Clinicians stress effect more than industry.Judging by effect: 44% of clinicians, 34% of industry.
How these are counted

Answer chart: of the 75 filers who took a side on this point (no 72, partially 2, yes 1). The answer chart counts only filers who named one option; proposal shares are of all who answered. Proposals: of the 198 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q2 →

What filers proposed

  1. Name specific output risk factors40%
  2. Judge by effect, not wording35%
  3. Disclaimers do not lower directiveness33%
  4. Publish worked examples and a scale23%
All 9 proposals
  1. Credit safeguards like sources and review13%
  2. Test how real users respond9%
  3. Assess whole conversations, not single replies9%
  4. Declare and test a directiveness ceiling8%
  5. Use a different test than directiveness5%
  1. Name specific output risk factors38%
  2. Judge by effect, not wording34%
  3. Disclaimers do not lower directiveness34%
  4. Publish worked examples and a scale25%
All 9 proposals
  1. Credit safeguards like sources and review17%
  2. Test how real users respond8%
  3. Assess whole conversations, not single replies8%
  4. Use a different test than directiveness7%
  5. Declare and test a directiveness ceiling6%
  1. Name specific output risk factors44%
  2. Judge by effect, not wording44%
  3. Disclaimers do not lower directiveness39%
  4. Publish worked examples and a scale22%
All 9 proposals
  1. Declare and test a directiveness ceiling15%
  2. Test how real users respond12%
  3. Credit safeguards like sources and review10%
  4. Assess whole conversations, not single replies10%
  5. Use a different test than directiveness0%

Only 13 filers in this group, so counts are shown, not shares.

  1. Judge by effect, not wording4
  2. Name specific output risk factors3
  3. Disclaimers do not lower directiveness3
  4. Publish worked examples and a scale2
All 9 proposals
  1. Test how real users respond2
  2. Credit safeguards like sources and review1
  3. Assess whole conversations, not single replies0
  4. Declare and test a directiveness ceiling0
  5. Use a different test than directiveness0

Only 3 filers in this group, so counts are shown, not shares.

  1. Disclaimers do not lower directiveness2
  2. Name specific output risk factors1
  3. Assess whole conversations, not single replies1
  4. Judge by effect, not wording0
All 9 proposals
  1. Publish worked examples and a scale0
  2. Credit safeguards like sources and review0
  3. Test how real users respond0
  4. Declare and test a directiveness ceiling0
  5. Use a different test than directiveness0
  1. Name specific output risk factors50%
  2. Judge by effect, not wording33%
  3. Disclaimers do not lower directiveness23%
  4. Publish worked examples and a scale20%
All 9 proposals
  1. Assess whole conversations, not single replies10%
  2. Credit safeguards like sources and review7%
  3. Test how real users respond7%
  4. Declare and test a directiveness ceiling7%
  5. Use a different test than directiveness3%

Do disclaimers reduce the risk of directive output?

96%
98%

Only 17 filers who took a side in this group, so counts are shown, not shares.

17

Only 5 filers who took a side in this group, so counts are shown, not shares.

41

Only 2 filers who took a side in this group, so counts are shown, not shares.

2

Only 10 filers who took a side in this group, so counts are shown, not shares.

91
NoPartiallyYes
Question 3 of 26 · Patients versus clinicians as users202 filers answered · 40% of all

When AI gives clinical information to patients instead of clinicians, does the risk change, and what safeguards help while still trusting patients with the information?

The FDA’s full question

CDRH seeks input on whether and when a GenAI-enabled function that results in delivery of clinical information to patients, as opposed to HCPs, could present different or higher risks, while also recognizing the potential benefits associated with improved patient empowerment, engagement, and access to clinical information. What device characteristics, output features, or safeguards might mitigate risks that could arise when a user lacks the domain knowledge to independently evaluate an output, without unnecessarily underestimating patient capability?

In the summary report: Takeaway 4, A disclaimer does not make AI advice safer, filers told the FDA

Key findingsFindings describe all filers.

  1. Risk depends on how the answer is built.Filers judge it by its safeguards, not as riskier by default.
  2. Several practical safeguards, used together.Reaching a person, testing with real patients, showing sources.
  3. Clinicians are warier of it than industry is.Calling it no riskier: industry 26%, clinicians 8%.
How these are counted

Answer chart: of the 202 filers who answered (depends 88, not riskier 36, always riskier 12, no side 66). Proposals: of the 202 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q3 →

Is patient-facing AI riskier?

44%18%33%
43%26%29%
53%8%12%27%

Only 10 filers in this group, so counts are shown, not shares.

3115

Only 4 filers in this group, so counts are shown, not shares.

112
39%8%8%44%
DependsNot riskierAlways riskierNo side

What filers proposed

  1. A route to a human or emergency care30%
  2. Testing with real patients25%
  3. Showing uncertainty and sources23%
  4. Credit for tested safeguards14%
All 6 proposals
  1. Telling patients it’s AI12%
  2. Clinician review before acting7%
  1. A route to a human or emergency care29%
  2. Testing with real patients29%
  3. Showing uncertainty and sources22%
  4. Credit for tested safeguards18%
All 6 proposals
  1. Telling patients it’s AI7%
  2. Clinician review before acting7%
  1. A route to a human or emergency care37%
  2. Showing uncertainty and sources22%
  3. Telling patients it’s AI22%
  4. Testing with real patients20%
All 6 proposals
  1. Clinician review before acting14%
  2. Credit for tested safeguards4%

Only 10 filers in this group, so counts are shown, not shares.

  1. Testing with real patients3
  2. A route to a human or emergency care2
  3. Showing uncertainty and sources2
  4. Credit for tested safeguards2
All 6 proposals
  1. Telling patients it’s AI0
  2. Clinician review before acting0

Only 4 filers in this group, so counts are shown, not shares.

  1. A route to a human or emergency care2
  2. Telling patients it’s AI2
  3. Testing with real patients1
  4. Credit for tested safeguards1
All 6 proposals
  1. Showing uncertainty and sources0
  2. Clinician review before acting0
  1. Showing uncertainty and sources28%
  2. A route to a human or emergency care22%
  3. Testing with real patients19%
  4. Credit for tested safeguards14%
All 6 proposals
  1. Telling patients it’s AI14%
  2. Clinician review before acting3%
Question 4 of 26 · Generalist and specialist users142 filers answered · 28% of all

Should it matter whether the clinician using the AI is a generalist or a specialist?

The FDA’s full question

CDRH seeks input on whether and how the distinction between generalist and specialist physicians might inform the assessment of risk for HCP-facing GenAI-enabled functions. Under what circumstances, if any, might risk be affected when an HCP who lacks the relevant clinical specialist knowledge receives information that falls within a specialist area of practice? What device characteristics or safeguards might mitigate such risks?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. Job title matters less than skill at the task.Risk turns on whether the user can check the output.
  2. Specialist AI for generalists divides filers.Some see added risk; others see wider access to expertise.
  3. Clinicians ask for more checks than industry.Testing with real users: clinicians 28%, industry 17%.
How these are counted

Asks chart: of the 178 specific asks linked to Q4 (more oversight 81, mixed 8, less oversight 11, neither: method or clarity 78). Proposals: of the 142 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q4 →

What filers proposed

  1. Judge risk by ability to check output28%
  2. Test with the real intended users22%
  3. Label the intended user and competency20%
  4. Flag specialist content and prompt referral20%
All 9 proposals
  1. Credit human review only when proven12%
  2. Do not presume generalist use riskier11%
  3. Have specialists help evaluate devices6%
  4. Include nurses and other clinicians6%
  5. Use existing processes, no new category4%
  1. Judge risk by ability to check output19%
  2. Test with the real intended users17%
  3. Label the intended user and competency17%
  4. Do not presume generalist use riskier15%
All 9 proposals
  1. Flag specialist content and prompt referral10%
  2. Credit human review only when proven10%
  3. Have specialists help evaluate devices6%
  4. Use existing processes, no new category6%
  5. Include nurses and other clinicians0%
  1. Judge risk by ability to check output35%
  2. Test with the real intended users28%
  3. Flag specialist content and prompt referral28%
  4. Label the intended user and competency22%
All 9 proposals
  1. Credit human review only when proven18%
  2. Include nurses and other clinicians12%
  3. Have specialists help evaluate devices10%
  4. Do not presume generalist use riskier8%
  5. Use existing processes, no new category0%

Only 5 filers in this group, so counts are shown, not shares.

  1. Test with the real intended users3
  2. Judge risk by ability to check output2
  3. Label the intended user and competency2
  4. Flag specialist content and prompt referral1
All 9 proposals
  1. Credit human review only when proven0
  2. Do not presume generalist use riskier0
  3. Have specialists help evaluate devices0
  4. Include nurses and other clinicians0
  5. Use existing processes, no new category0

Only 1 filer in this group, so counts are shown, not shares.

  1. Flag specialist content and prompt referral1
  2. Judge risk by ability to check output0
  3. Test with the real intended users0
  4. Label the intended user and competency0
All 9 proposals
  1. Credit human review only when proven0
  2. Do not presume generalist use riskier0
  3. Have specialists help evaluate devices0
  4. Include nurses and other clinicians0
  5. Use existing processes, no new category0
  1. Judge risk by ability to check output42%
  2. Flag specialist content and prompt referral38%
  3. Label the intended user and competency25%
  4. Test with the real intended users21%
All 9 proposals
  1. Credit human review only when proven12%
  2. Include nurses and other clinicians12%
  3. Do not presume generalist use riskier8%
  4. Use existing processes, no new category4%
  5. Have specialists help evaluate devices0%

Which way do the asks push?

46%44%
41%11%42%
48%47%

Only 6 asks in this group, so counts are shown, not shares.

42

Only 1 asks in this group, so counts are shown, not shares.

1
50%44%
More oversightMixedLess oversightNeither: method or clarity
Question 5 of 26 · Multi-turn conversations that migrate189 filers answered · 38% of all

How is risk assessed when a conversation starts with non-directive information and drifts into action-directing?

The FDA’s full question

For multi-turn conversational GenAI-enabled devices that may migrate from providing “non-directive” information to “action-directing” information over the course of an exchange, how should risk be assessed across realistic conversational trajectories? How could the intended use of such a device be characterized when its behavior is emergent across a conversation?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. The whole conversation is the unit of risk.Not reply by reply; a majority in each group of 20 or more.
  2. Filers split on how to classify drift.By the most directive point reachable, or by the declared use.
  3. Industry favors declared limits and testing.A bounded scope and ceiling: industry 38%, clinicians 23%.
How these are counted

Asks chart: of the 261 specific asks linked to Q5 (more oversight 144, mixed 4, less oversight 5, neither: method or clarity 108). Proposals: of the 189 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q5 →

What filers proposed

  1. Judge risk across the whole conversation64%
  2. Declare a bounded scope and ceiling31%
  3. Test with pressure and adversarial users22%
  4. Build controls that stop drift20%
All 8 proposals
  1. Test long conversations and repeat sessions17%
  2. Rate risk by highest point reached16%
  3. Log conversations and monitor after launch11%
  4. Keep intended use set by maker5%
  1. Judge risk across the whole conversation65%
  2. Declare a bounded scope and ceiling38%
  3. Test with pressure and adversarial users31%
  4. Build controls that stop drift23%
All 8 proposals
  1. Test long conversations and repeat sessions19%
  2. Rate risk by highest point reached14%
  3. Log conversations and monitor after launch12%
  4. Keep intended use set by maker7%
  1. Judge risk across the whole conversation67%
  2. Declare a bounded scope and ceiling23%
  3. Rate risk by highest point reached23%
  4. Test long conversations and repeat sessions21%
All 8 proposals
  1. Test with pressure and adversarial users18%
  2. Build controls that stop drift18%
  3. Log conversations and monitor after launch10%
  4. Keep intended use set by maker3%

Only 16 filers in this group, so counts are shown, not shares.

  1. Judge risk across the whole conversation11
  2. Declare a bounded scope and ceiling3
  3. Test long conversations and repeat sessions3
  4. Rate risk by highest point reached3
All 8 proposals
  1. Test with pressure and adversarial users1
  2. Build controls that stop drift1
  3. Log conversations and monitor after launch1
  4. Keep intended use set by maker0

Only 3 filers in this group, so counts are shown, not shares.

  1. Judge risk across the whole conversation1
  2. Test with pressure and adversarial users1
  3. Rate risk by highest point reached1
  4. Declare a bounded scope and ceiling0
All 8 proposals
  1. Build controls that stop drift0
  2. Test long conversations and repeat sessions0
  3. Log conversations and monitor after launch0
  4. Keep intended use set by maker0
  1. Judge risk across the whole conversation59%
  2. Declare a bounded scope and ceiling24%
  3. Build controls that stop drift24%
  4. Rate risk by highest point reached14%
All 8 proposals
  1. Test long conversations and repeat sessions10%
  2. Log conversations and monitor after launch10%
  3. Test with pressure and adversarial users3%
  4. Keep intended use set by maker3%

Which way do the asks push?

55%41%
53%42%
59%39%

Only 18 asks in this group, so counts are shown, not shares.

117

Only 3 asks in this group, so counts are shown, not shares.

21
56%44%
More oversightMixedLess oversightNeither: method or clarity
Question 6 of 26 · Care escalation functions173 filers answered · 34% of all

How should under-escalation be weighed against over-escalation?

The FDA’s full question

For GenAI-enabled care escalation functions, CDRH is considering how the evaluation may account for both under-escalation and over-escalation. How could manufacturers characterize and weigh these two directions of error, given that they may not be commensurable and that acceptable trade-offs may vary by clinical context?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. Two errors, two different harms.Filers want each measured and reported, not one accuracy score.
  2. No single weighting rule fits every device.The largest group would set the balance by clinical context.
  3. Clinicians weigh missed escalation more.Weighing it more: 5 of 23 clinicians, 11 of 71 industry.
How these are counted

Answer chart: of the 119 filers who took a side on this point (by clinical context 56, report each separately 41, missed escalation weighs more 17, more than two error types 5). The answer chart counts only filers who named one option; proposal shares are of all who answered. Proposals: of the 173 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q6 →

What filers proposed

  1. Report each error type separately58%
  2. Sponsors justify trade-off in advance42%
  3. Put missed escalation first21%
  4. Break results down by acuity and group13%
All 9 proposals
  1. No single FDA-wide weighting rule12%
  2. Keep monitoring errors after launch10%
  3. Check what happens after escalation8%
  4. Clinicians and patients help set thresholds7%
  5. Count more than two error types7%
  1. Report each error type separately62%
  2. Sponsors justify trade-off in advance50%
  3. Put missed escalation first23%
  4. No single FDA-wide weighting rule13%
All 9 proposals
  1. Break results down by acuity and group12%
  2. Keep monitoring errors after launch9%
  3. Check what happens after escalation7%
  4. Count more than two error types5%
  5. Clinicians and patients help set thresholds3%
  1. Report each error type separately52%
  2. Sponsors justify trade-off in advance35%
  3. Put missed escalation first25%
  4. Break results down by acuity and group12%
All 9 proposals
  1. No single FDA-wide weighting rule12%
  2. Clinicians and patients help set thresholds12%
  3. Keep monitoring errors after launch10%
  4. Check what happens after escalation10%
  5. Count more than two error types8%

Only 14 filers in this group, so counts are shown, not shares.

  1. Report each error type separately9
  2. Sponsors justify trade-off in advance4
  3. Keep monitoring errors after launch3
  4. Check what happens after escalation2
All 9 proposals
  1. Count more than two error types2
  2. Break results down by acuity and group1
  3. No single FDA-wide weighting rule1
  4. Clinicians and patients help set thresholds1
  5. Put missed escalation first0

Only 1 filer in this group, so counts are shown, not shares.

  1. No single FDA-wide weighting rule1
  2. Report each error type separately0
  3. Sponsors justify trade-off in advance0
  4. Put missed escalation first0
All 9 proposals
  1. Break results down by acuity and group0
  2. Keep monitoring errors after launch0
  3. Check what happens after escalation0
  4. Clinicians and patients help set thresholds0
  5. Count more than two error types0
  1. Report each error type separately50%
  2. Sponsors justify trade-off in advance35%
  3. Put missed escalation first19%
  4. Break results down by acuity and group19%
All 9 proposals
  1. Keep monitoring errors after launch12%
  2. Clinicians and patients help set thresholds12%
  3. No single FDA-wide weighting rule8%
  4. Count more than two error types8%
  5. Check what happens after escalation4%

How should the two errors be weighed?

47%34%14%
54%30%15%
30%43%22%

Only 9 filers who took a side in this group, so counts are shown, not shares.

342

Only 1 filer who took a side in this group, so counts are shown, not shares.

1

Only 15 filers who took a side in this group, so counts are shown, not shares.

76
By clinical contextReport each separatelyMissed escalation weighs moreMore than two error types
Question 7 of 26 · The competency-based approach273 filers answered · 54% of all

Is the two-step approach, benchmark the AI, then confirm it in clinical use, the right way to evaluate these devices?

The FDA’s full question

Is the competency-based approach described above, i.e., device benchmarking followed by clinical confirmation, a useful and appropriate framework for evaluating GenAI-enabled devices?

In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes

Key findingsFindings describe all filers.

  1. Support for the two steps is broad and even.Each group of 20 or more filers backs it at about the same rate.
  2. Asks for changes point to stricter testing.Testing the exact device as deployed, and again after launch.
  3. Researchers doubt benchmarks on their own.Real clinical evidence: researchers 24%, industry 9%.
How these are counted

Answer chart: of the 273 filers who answered (accept, with changes 134, accept, as written 57, split 1, reject 3, no verdict 78). Proposals: of the 273 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q7 →

Is the two-step approach the right framework?

49%21%29%
49%21%27%
58%14%27%
48%24%28%

Only 3 filers in this group, so counts are shown, not shares.

12
40%28%32%
Accept, with changesAccept, as writtenSplitRejectAnswered, no verdict

Of the 195 who gave a verdict, 134 accepted with changes, 57 accepted as written, 1 was split and 3 rejected.

Of the 106 who gave a verdict, 72 accepted with changes, 31 accepted as written, 1 was split and 2 rejected.

Of the 43 who gave a verdict, 34 accepted with changes, 8 accepted as written and 1 rejected.

Of the 18 who gave a verdict, 12 accepted with changes and 6 accepted as written.

The one filer who gave a verdict accepted as written.

Of the 27 who gave a verdict, 16 accepted with changes and 11 accepted as written.

What filers proposed

  1. Test the exact deployed configuration16%
  2. Keep re-testing after market authorization12%
  3. Require real clinical evidence, not just benchmarks12%
  4. Test clinicians and AI together in workflow9%
All 9 proposals
  1. Keep existing validation rules and terms9%
  2. Define competency by specific task and claim8%
  3. Expand device autonomy in supervised stages5%
  4. Use independent reviewers and protected tests4%
  5. Include nurses and other actual users4%
  1. Test the exact deployed configuration16%
  2. Keep existing validation rules and terms10%
  3. Keep re-testing after market authorization10%
  4. Require real clinical evidence, not just benchmarks9%
All 9 proposals
  1. Test clinicians and AI together in workflow8%
  2. Define competency by specific task and claim8%
  3. Expand device autonomy in supervised stages5%
  4. Use independent reviewers and protected tests3%
  5. Include nurses and other actual users3%
  1. Test the exact deployed configuration14%
  2. Keep existing validation rules and terms14%
  3. Require real clinical evidence, not just benchmarks12%
  4. Test clinicians and AI together in workflow12%
All 9 proposals
  1. Keep re-testing after market authorization10%
  2. Define competency by specific task and claim10%
  3. Include nurses and other actual users7%
  4. Expand device autonomy in supervised stages5%
  5. Use independent reviewers and protected tests5%
  1. Require real clinical evidence, not just benchmarks24%
  2. Test the exact deployed configuration8%
  3. Test clinicians and AI together in workflow8%
  4. Expand device autonomy in supervised stages8%
All 9 proposals
  1. Keep re-testing after market authorization4%
  2. Define competency by specific task and claim4%
  3. Use independent reviewers and protected tests4%
  4. Include nurses and other actual users4%
  5. Keep existing validation rules and terms0%

Only 3 filers in this group, so counts are shown, not shares.

  1. Keep re-testing after market authorization1
  2. Require real clinical evidence, not just benchmarks1
  3. Include nurses and other actual users1
  4. Test the exact deployed configuration0
All 9 proposals
  1. Test clinicians and AI together in workflow0
  2. Keep existing validation rules and terms0
  3. Define competency by specific task and claim0
  4. Expand device autonomy in supervised stages0
  5. Use independent reviewers and protected tests0
  1. Keep re-testing after market authorization28%
  2. Test the exact deployed configuration25%
  3. Require real clinical evidence, not just benchmarks12%
  4. Define competency by specific task and claim12%
All 9 proposals
  1. Use independent reviewers and protected tests10%
  2. Test clinicians and AI together in workflow8%
  3. Keep existing validation rules and terms2%
  4. Expand device autonomy in supervised stages2%
  5. Include nurses and other actual users2%
Question 8 of 26 · Mapping the risk grid to evidence176 filers answered · 35% of all

Should an AI device’s position on the risk map help decide how much evidence it must bring before market?

The FDA’s full question

How might the two-axis risk framework described in Section IV be considered within the competency-based approach described above to help determine the level of evidence needed for a premarket submission?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. Evidence should rise with risk position.Clinicians back this more often: 92% against 73% of industry.
  2. Industry asks more often for a published map.A table or worked examples: industry 30%, clinicians 6%.
  3. Clinicians' asks lean toward more oversight.More oversight: 55% of clinicians' asks, 26% of industry's.
How these are counted

Asks chart: of the 225 specific asks linked to Q8 (more oversight 80, mixed 17, less oversight 21, neither: method or clarity 107). Proposals: of the 176 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q8 →

What filers proposed

  1. Scale evidence to risk position79%
  2. Publish a risk to evidence map21%
  3. Add risk factors beyond two axes20%
  4. Lighter evidence for low-risk functions11%
All 9 proposals
  1. Prospective studies for highest risk9%
  2. Credit safeguards only when proven9%
  3. Let sponsors justify their evidence7%
  4. Rely on existing risk management processes6%
  5. Keep a minimum evidence floor6%
  1. Scale evidence to risk position73%
  2. Publish a risk to evidence map30%
  3. Add risk factors beyond two axes13%
  4. Lighter evidence for low-risk functions11%
All 9 proposals
  1. Let sponsors justify their evidence10%
  2. Prospective studies for highest risk9%
  3. Credit safeguards only when proven8%
  4. Rely on existing risk management processes8%
  5. Keep a minimum evidence floor6%
  1. Scale evidence to risk position92%
  2. Add risk factors beyond two axes39%
  3. Lighter evidence for low-risk functions14%
  4. Prospective studies for highest risk14%
All 9 proposals
  1. Credit safeguards only when proven14%
  2. Publish a risk to evidence map6%
  3. Keep a minimum evidence floor3%
  4. Let sponsors justify their evidence0%
  5. Rely on existing risk management processes0%

Only 15 filers in this group, so counts are shown, not shares.

  1. Scale evidence to risk position13
  2. Add risk factors beyond two axes3
  3. Publish a risk to evidence map2
  4. Rely on existing risk management processes2
All 9 proposals
  1. Lighter evidence for low-risk functions1
  2. Prospective studies for highest risk1
  3. Keep a minimum evidence floor1
  4. Credit safeguards only when proven0
  5. Let sponsors justify their evidence0

Only 2 filers in this group, so counts are shown, not shares.

  1. Scale evidence to risk position2
  2. Publish a risk to evidence map0
  3. Add risk factors beyond two axes0
  4. Lighter evidence for low-risk functions0
All 9 proposals
  1. Prospective studies for highest risk0
  2. Credit safeguards only when proven0
  3. Let sponsors justify their evidence0
  4. Rely on existing risk management processes0
  5. Keep a minimum evidence floor0
  1. Scale evidence to risk position79%
  2. Add risk factors beyond two axes25%
  3. Publish a risk to evidence map12%
  4. Lighter evidence for low-risk functions12%
All 9 proposals
  1. Credit safeguards only when proven8%
  2. Let sponsors justify their evidence8%
  3. Keep a minimum evidence floor8%
  4. Prospective studies for highest risk4%
  5. Rely on existing risk management processes4%

Which way do the asks push?

36%9%48%
26%10%11%54%
55%8%31%

Only 12 asks in this group, so counts are shown, not shares.

714

Only 1 asks in this group, so counts are shown, not shares.

1
41%52%
More oversightMixedLess oversightNeither: method or clarity
Question 9 of 26 · The benchmarking structure255 filers answered · 51% of all

Do the ten benchmark competencies, from clinical knowledge to generalizability, add up to enough evidence of safety and effectiveness?

The FDA’s full question

Would a benchmarking structure such as the one described above be likely to provide adequate evidence of clinical knowledge, analytic capabilities, safety behavior, communication, and generalizability to support a reasonable assurance of safety and effectiveness? Are there elements that are missing, redundant, or inappropriately categorized? Are there externally developed standards that could be leveraged?

In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes

Key findingsFindings describe all filers.

  1. Filers keep the ten and ask for more.Nearly all who change the list add to it rather than cut it.
  2. Each group asks equally for outside standards.Industry, clinicians and researchers name them at the same rate.
  3. Researchers ask for results by patient group.Researchers 33% against 5% of industry; clinicians fall between.
How these are counted

Answer chart: of the 255 filers who answered (accept, with changes 61, accept, as written 3, split 0, reject 0, no verdict 191). Proposals: of the 255 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q9 →

Do the ten competencies add up to enough evidence?

24%75%
24%74%
33%65%
10%90%

Only 3 filers in this group, so counts are shown, not shares.

12
18%82%
Accept, with changesAccept, as writtenAnswered, no verdict

Of the 64 who gave a verdict, 61 accepted with changes and 3 accepted as written.

Of the 34 who gave a verdict, 32 accepted with changes and 2 accepted as written.

Of the 18 who gave a verdict, 17 accepted with changes and 1 accepted as written.

Both who gave a verdict accepted with changes.

The one filer who gave a verdict accepted with changes.

All 9 who gave a verdict accepted with changes.

What filers proposed

  1. Use outside standards and specialty bodies22%
  2. Check outputs against their sources17%
  3. Report results for more groups13%
  4. Test with real intended users11%
All 9 proposals
  1. Test long and repeat conversations9%
  2. Test security and prompt injection9%
  3. Repeat runs and report variability8%
  4. Verify safeguards work outside the model8%
  5. Separate pass limits for serious failures6%
  1. Use outside standards and specialty bodies22%
  2. Check outputs against their sources21%
  3. Test long and repeat conversations11%
  4. Test security and prompt injection11%
All 9 proposals
  1. Verify safeguards work outside the model10%
  2. Test with real intended users8%
  3. Repeat runs and report variability8%
  4. Report results for more groups5%
  5. Separate pass limits for serious failures5%
  1. Use outside standards and specialty bodies22%
  2. Report results for more groups22%
  3. Check outputs against their sources16%
  4. Test with real intended users14%
All 9 proposals
  1. Repeat runs and report variability14%
  2. Separate pass limits for serious failures14%
  3. Test long and repeat conversations6%
  4. Test security and prompt injection4%
  5. Verify safeguards work outside the model4%
  1. Report results for more groups33%
  2. Use outside standards and specialty bodies19%
  3. Check outputs against their sources14%
  4. Test with real intended users14%
All 9 proposals
  1. Verify safeguards work outside the model14%
  2. Test long and repeat conversations10%
  3. Test security and prompt injection10%
  4. Separate pass limits for serious failures5%
  5. Repeat runs and report variability0%

Only 3 filers in this group, so counts are shown, not shares.

  1. Report results for more groups2
  2. Test with real intended users1
  3. Use outside standards and specialty bodies0
  4. Check outputs against their sources0
All 9 proposals
  1. Test long and repeat conversations0
  2. Test security and prompt injection0
  3. Repeat runs and report variability0
  4. Verify safeguards work outside the model0
  5. Separate pass limits for serious failures0
  1. Use outside standards and specialty bodies24%
  2. Report results for more groups16%
  3. Check outputs against their sources10%
  4. Test with real intended users10%
All 9 proposals
  1. Test long and repeat conversations6%
  2. Test security and prompt injection6%
  3. Repeat runs and report variability4%
  4. Verify safeguards work outside the model4%
  5. Separate pass limits for serious failures2%
Question 10 of 26 · Benchmark contamination and saturation223 filers answered · 44% of all

How can a benchmark score be shown to predict real-world behavior?

The FDA’s full question

CDRH recognizes that publicly available benchmarking assets may be subject to data contamination, saturation, and limited real-world representativeness. How should a sponsor establish that performance on a given benchmark predicts safe and effective real-world behavior for the device’s intended use? What evidence should support the construct validity of a benchmark used to gate device evaluation, and what role should sponsor-developed benchmarks play given potential concerns around independence and optimization to the test?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. A score must be shown to fit the intended use.Filers do not presume a benchmark predicts clinical behavior.
  2. Sponsor tests are acceptable, with safeguards.Such as test data kept hidden from the developer.
  3. Researchers press for real-world proof.Proof scores predict results: researchers 42%, industry 24%.
How these are counted

Asks chart: of the 371 specific asks linked to Q10 (more oversight 267, mixed 3, less oversight 4, neither: method or clarity 97). Proposals: of the 223 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q10 →

What filers proposed

  1. Allow sponsor benchmarks with safeguards44%
  2. Keep test data hidden from developers43%
  3. Prove scores predict real-world results30%
  4. Show benchmark matches intended use26%
All 9 proposals
  1. Lock test plans and criteria in advance19%
  2. Check for training data contamination17%
  3. Refresh, rotate or retire benchmarks16%
  4. Build shared recognized benchmarks13%
  5. Let sponsors pick fit-for-purpose tests6%
  1. Allow sponsor benchmarks with safeguards46%
  2. Keep test data hidden from developers44%
  3. Show benchmark matches intended use27%
  4. Prove scores predict real-world results24%
All 9 proposals
  1. Lock test plans and criteria in advance22%
  2. Refresh, rotate or retire benchmarks22%
  3. Check for training data contamination18%
  4. Build shared recognized benchmarks14%
  5. Let sponsors pick fit-for-purpose tests8%
  1. Keep test data hidden from developers44%
  2. Allow sponsor benchmarks with safeguards41%
  3. Show benchmark matches intended use39%
  4. Prove scores predict real-world results34%
All 9 proposals
  1. Lock test plans and criteria in advance22%
  2. Check for training data contamination15%
  3. Refresh, rotate or retire benchmarks15%
  4. Build shared recognized benchmarks10%
  5. Let sponsors pick fit-for-purpose tests7%
  1. Allow sponsor benchmarks with safeguards42%
  2. Keep test data hidden from developers42%
  3. Prove scores predict real-world results42%
  4. Lock test plans and criteria in advance17%
All 9 proposals
  1. Show benchmark matches intended use12%
  2. Check for training data contamination12%
  3. Build shared recognized benchmarks8%
  4. Refresh, rotate or retire benchmarks4%
  5. Let sponsors pick fit-for-purpose tests0%

Only 1 filer in this group, so counts are shown, not shares.

  1. Allow sponsor benchmarks with safeguards1
  2. Keep test data hidden from developers1
  3. Prove scores predict real-world results0
  4. Show benchmark matches intended use0
All 9 proposals
  1. Lock test plans and criteria in advance0
  2. Check for training data contamination0
  3. Refresh, rotate or retire benchmarks0
  4. Build shared recognized benchmarks0
  5. Let sponsors pick fit-for-purpose tests0
  1. Allow sponsor benchmarks with safeguards42%
  2. Keep test data hidden from developers36%
  3. Prove scores predict real-world results36%
  4. Show benchmark matches intended use18%
All 9 proposals
  1. Check for training data contamination18%
  2. Build shared recognized benchmarks15%
  3. Lock test plans and criteria in advance6%
  4. Refresh, rotate or retire benchmarks6%
  5. Let sponsors pick fit-for-purpose tests0%

Which way do the asks push?

72%26%
67%31%
77%23%
84%16%

Only 1 asks in this group, so counts are shown, not shares.

1
76%20%
More oversightMixedLess oversightNeither: method or clarity
Question 11 of 26 · Clinical confirmation without a prospective trial209 filers answered · 41% of all

When can a device be confirmed without a prospective clinical study, and what earns that lighter path?

The FDA’s full question

CDRH is considering that clinical confirmation for a GenAI-enabled device might not require a prospective clinical study in every case, and has described above a range of approaches of increasing rigor and patient exposure. How might a sponsor select and justify a confirmation approach tailored to a device’s intended use and proportionate to the device’s risk profile? Are there device types or risk profiles for which one or more of these approaches would be insufficient or inappropriate? How might the anticipated distribution of real-world inputs be taken into consideration? Are there other methods of clinical confirmation that might help inform the evaluation of GenAI-enabled devices?

In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes

Key findingsFindings describe all filers.

  1. Few insist on a new trial for every device.The debate is where the line for a prospective study falls.
  2. High-risk functions should still get a study.Autonomous, irreversible or high-consequence uses.
  3. Industry accepts the approach more readily.Industry: 30% with changes, 10% as written. Clinicians: 26% and 2%.
How these are counted

Answer chart: of the 209 filers who answered (accept, with changes 57, accept, as written 13, split 1, reject 0, no verdict 138). Proposals: of the 209 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q11 →

Do filers accept the FDA’s confirmation approach?

27%66%
30%10%59%
26%72%

Only 19 filers in this group, so counts are shown, not shares.

18

Only 2 filers in this group, so counts are shown, not shares.

11
31%66%
Accept, with changesAccept, as writtenSplitAnswered, no verdict

Of the 71 who gave a verdict, 57 accepted with changes, 13 accepted as written and 1 was split.

Of the 46 who gave a verdict, 34 accepted with changes, 11 accepted as written and 1 was split.

Of the 13 who gave a verdict, 12 accepted with changes and 1 accepted as written.

The one filer who gave a verdict accepted with changes.

The one filer who gave a verdict accepted with changes.

Of the 10 who gave a verdict, 9 accepted with changes and 1 accepted as written.

What filers proposed

  1. Prospective studies for highest-risk functions26%
  2. Test how real users act on outputs20%
  3. Justify the method by remaining uncertainty20%
  4. Cover real-world inputs and rare cases17%
All 9 proposals
  1. Accept real-world, local and sandbox data14%
  2. Count staged real-world rollout as evidence11%
  3. Publish a guide matching risk to method9%
  4. Accept simulation and synthetic testing9%
  5. Set rules for evidence from other settings5%
  1. Prospective studies for highest-risk functions21%
  2. Justify the method by remaining uncertainty21%
  3. Test how real users act on outputs14%
  4. Accept real-world, local and sandbox data13%
All 9 proposals
  1. Publish a guide matching risk to method13%
  2. Cover real-world inputs and rare cases11%
  3. Count staged real-world rollout as evidence11%
  4. Accept simulation and synthetic testing10%
  5. Set rules for evidence from other settings5%
  1. Prospective studies for highest-risk functions43%
  2. Cover real-world inputs and rare cases28%
  3. Test how real users act on outputs26%
  4. Justify the method by remaining uncertainty17%
All 9 proposals
  1. Accept real-world, local and sandbox data17%
  2. Count staged real-world rollout as evidence13%
  3. Accept simulation and synthetic testing11%
  4. Set rules for evidence from other settings4%
  5. Publish a guide matching risk to method0%

Only 19 filers in this group, so counts are shown, not shares.

  1. Test how real users act on outputs8
  2. Cover real-world inputs and rare cases5
  3. Justify the method by remaining uncertainty4
  4. Prospective studies for highest-risk functions3
All 9 proposals
  1. Set rules for evidence from other settings3
  2. Accept real-world, local and sandbox data2
  3. Count staged real-world rollout as evidence2
  4. Accept simulation and synthetic testing1
  5. Publish a guide matching risk to method0

Only 2 filers in this group, so counts are shown, not shares.

  1. Test how real users act on outputs2
  2. Cover real-world inputs and rare cases1
  3. Prospective studies for highest-risk functions0
  4. Justify the method by remaining uncertainty0
All 9 proposals
  1. Accept real-world, local and sandbox data0
  2. Count staged real-world rollout as evidence0
  3. Publish a guide matching risk to method0
  4. Accept simulation and synthetic testing0
  5. Set rules for evidence from other settings0
  1. Prospective studies for highest-risk functions28%
  2. Justify the method by remaining uncertainty21%
  3. Cover real-world inputs and rare cases17%
  4. Accept real-world, local and sandbox data17%
All 9 proposals
  1. Test how real users act on outputs14%
  2. Publish a guide matching risk to method14%
  3. Count staged real-world rollout as evidence7%
  4. Accept simulation and synthetic testing7%
  5. Set rules for evidence from other settings0%
Question 12 of 26 · Statistically meaningful performance154 filers answered · 31% of all

How do you get statistically meaningful performance numbers when synthetic inputs are mixed with real ones?

The FDA’s full question

How should sponsors achieve statistically meaningful performance measurement for GenAI-enabled devices? Where synthetically generated inputs supplement real patient data, how should sponsors account for differences between the synthetic and real-world distributions when estimating performance, and under what conditions, if any, is it appropriate to combine benchmarking evidence and clinical confirmation evidence to support a single performance estimate?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. Filers want synthetic and real results apart.Pooling only where the two are shown to be comparable.
  2. Acceptance read on the bound, not the average.Confidence intervals, repeat runs, endpoints set early.
  3. Clinicians focus on rare, serious failures.Testing for them, not averages: clinicians 28%, industry 19%.
How these are counted

Asks chart: of the 257 specific asks linked to Q12 (more oversight 142, mixed 1, less oversight 3, neither: method or clarity 111). Proposals: of the 154 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q12 →

What filers proposed

  1. Report synthetic and real results separately40%
  2. Report uncertainty and judge on bounds32%
  3. Prove synthetic data resembles real patients25%
  4. Set endpoints and sample sizes beforehand25%
All 9 proposals
  1. Run tests repeatedly and report variation19%
  2. Target rare serious failures, not averages18%
  3. Keep benchmark and clinical evidence apart16%
  4. Report results for patient subgroups13%
  5. Leave methods and combining flexible6%
  1. Report synthetic and real results separately39%
  2. Report uncertainty and judge on bounds28%
  3. Set endpoints and sample sizes beforehand26%
  4. Prove synthetic data resembles real patients22%
All 9 proposals
  1. Run tests repeatedly and report variation20%
  2. Keep benchmark and clinical evidence apart20%
  3. Target rare serious failures, not averages19%
  4. Report results for patient subgroups13%
  5. Leave methods and combining flexible7%
  1. Report synthetic and real results separately48%
  2. Report uncertainty and judge on bounds48%
  3. Set endpoints and sample sizes beforehand28%
  4. Target rare serious failures, not averages28%
All 9 proposals
  1. Report results for patient subgroups28%
  2. Prove synthetic data resembles real patients20%
  3. Run tests repeatedly and report variation16%
  4. Keep benchmark and clinical evidence apart8%
  5. Leave methods and combining flexible4%
  1. Report synthetic and real results separately43%
  2. Report uncertainty and judge on bounds38%
  3. Prove synthetic data resembles real patients38%
  4. Run tests repeatedly and report variation24%
All 9 proposals
  1. Set endpoints and sample sizes beforehand19%
  2. Target rare serious failures, not averages14%
  3. Keep benchmark and clinical evidence apart10%
  4. Report results for patient subgroups5%
  5. Leave methods and combining flexible0%

Only 1 filer in this group, so counts are shown, not shares.

  1. Report uncertainty and judge on bounds1
  2. Set endpoints and sample sizes beforehand1
  3. Report synthetic and real results separately0
  4. Prove synthetic data resembles real patients0
All 9 proposals
  1. Run tests repeatedly and report variation0
  2. Target rare serious failures, not averages0
  3. Keep benchmark and clinical evidence apart0
  4. Report results for patient subgroups0
  5. Leave methods and combining flexible0
  1. Report synthetic and real results separately36%
  2. Prove synthetic data resembles real patients32%
  3. Report uncertainty and judge on bounds23%
  4. Set endpoints and sample sizes beforehand18%
All 9 proposals
  1. Keep benchmark and clinical evidence apart18%
  2. Run tests repeatedly and report variation14%
  3. Target rare serious failures, not averages9%
  4. Leave methods and combining flexible9%
  5. Report results for patient subgroups5%

Which way do the asks push?

55%43%
55%43%
61%33%
56%44%

Only 3 asks in this group, so counts are shown, not shares.

12
50%50%
More oversightMixedLess oversightNeither: method or clarity
Question 13 of 26 · Synthetic data164 filers answered · 33% of all

Where is synthetic data good enough, and where is it not?

The FDA’s full question

For which clinical domains, device functions, or subpopulations is synthetic data particularly well-suited, or particularly inadequate, as a supplement to real-world evidence? What safeguards would mitigate the risk that synthetic data generated by models of the same class as the device under evaluation reproduces the very performance gaps the evaluation is intended to detect, particularly for underrepresented subgroups?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. Synthetic data fits rare and stress cases.It adds to real patient data and does not replace it.
  2. Synthetic data for subgroups divides filers.Some see wider coverage; others want real data for claims.
  3. Industry stresses an independent generator.Another model family: industry 37%, clinicians 21%.
How these are counted

Asks chart: of the 186 specific asks linked to Q13 (more oversight 139, mixed 1, less oversight 2, neither: method or clarity 44). Proposals: of the 164 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q13 →

What filers proposed

  1. Use it for rare cases and stress tests55%
  2. Supplement, never replace, real data37%
  3. Check synthetic results against real data34%
  4. Generate from a different model family30%
All 9 proposals
  1. Disclose how synthetic data was made23%
  2. Base subgroup claims on real data17%
  3. Have clinicians review synthetic cases16%
  4. Report synthetic and real results separately11%
  5. Leave the approach flexible and risk based5%
  1. Use it for rare cases and stress tests56%
  2. Generate from a different model family37%
  3. Supplement, never replace, real data35%
  4. Check synthetic results against real data33%
All 9 proposals
  1. Have clinicians review synthetic cases22%
  2. Base subgroup claims on real data17%
  3. Disclose how synthetic data was made13%
  4. Report synthetic and real results separately8%
  5. Leave the approach flexible and risk based7%
  1. Use it for rare cases and stress tests56%
  2. Supplement, never replace, real data44%
  3. Check synthetic results against real data32%
  4. Disclose how synthetic data was made26%
All 9 proposals
  1. Generate from a different model family21%
  2. Base subgroup claims on real data18%
  3. Report synthetic and real results separately18%
  4. Have clinicians review synthetic cases15%
  5. Leave the approach flexible and risk based3%

Only 17 filers in this group, so counts are shown, not shares.

  1. Use it for rare cases and stress tests8
  2. Check synthetic results against real data7
  3. Report synthetic and real results separately4
  4. Generate from a different model family3
All 9 proposals
  1. Base subgroup claims on real data3
  2. Supplement, never replace, real data2
  3. Disclose how synthetic data was made2
  4. Have clinicians review synthetic cases0
  5. Leave the approach flexible and risk based0

Only 2 filers in this group, so counts are shown, not shares.

  1. Supplement, never replace, real data2
  2. Use it for rare cases and stress tests1
  3. Disclose how synthetic data was made1
  4. Check synthetic results against real data0
All 9 proposals
  1. Generate from a different model family0
  2. Base subgroup claims on real data0
  3. Have clinicians review synthetic cases0
  4. Report synthetic and real results separately0
  5. Leave the approach flexible and risk based0
  1. Use it for rare cases and stress tests59%
  2. Disclose how synthetic data was made59%
  3. Supplement, never replace, real data45%
  4. Check synthetic results against real data41%
All 9 proposals
  1. Generate from a different model family32%
  2. Base subgroup claims on real data18%
  3. Have clinicians review synthetic cases9%
  4. Report synthetic and real results separately5%
  5. Leave the approach flexible and risk based5%

Which way do the asks push?

75%24%
71%27%
71%27%

Only 17 asks in this group, so counts are shown, not shares.

152

Only 1 asks in this group, so counts are shown, not shares.

1
83%17%
More oversightMixedLess oversightNeither: method or clarity
Question 14 of 26 · Comparators and acceptance criteria241 filers answered · 48% of all

For open-ended AI outputs, who is the performance comparator: a clinician panel, generalists, specialists, or the human-AI team?

The FDA’s full question

For open-ended device outputs, how should performance comparators and acceptance criteria be selected? When a panel of qualified clinicians serves as the comparator, how should the applicable standard (for example, the standard of care versus the performance of a median clinician in practice) be defined and justified? Should generalist or specialist physicians be used as a performance standard? When should human-AI team performance, rather than the device operating alone, serve as the basis for evaluation?

In the summary report: Takeaway 9, No single standard emerged for measuring AI devices

Key findingsFindings describe all filers.

  1. No agreed default: fit it to each device.The FDA’s own example, the median clinician, drew 2 of 129.
  2. Clinician and AI tested together.A majority in every kind of filer with 20 or more answers.
  3. Clinicians want a higher bar than industry.An evidence-based standard: 7 of 36 clinicians, 2 of 68 industry.
How these are counted

Answer chart: of the 129 filers who took a side on this point (tiered or several 46, match the real workflow 34, care without the device 25, specialist panel 11, evidence-based standard 11, median clinician 2). The answer chart counts only filers who named one option; proposal shares are of all who answered. Proposals: of the 241 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q14 →

What filers proposed

  1. Test clinician and AI together59%
  2. Match comparator to intended use39%
  3. Also report device-alone performance24%
  4. Use specialists for specialist tasks20%
All 9 proposals
  1. Compare against care without the device15%
  2. Do not let average clinicians set bar15%
  3. Report panel agreement and makeup15%
  4. Safety minimums that cannot be offset9%
  5. Let sponsors choose and justify comparators5%
  1. Test clinician and AI together54%
  2. Match comparator to intended use35%
  3. Also report device-alone performance23%
  4. Report panel agreement and makeup19%
All 9 proposals
  1. Use specialists for specialist tasks19%
  2. Compare against care without the device19%
  3. Do not let average clinicians set bar14%
  4. Safety minimums that cannot be offset8%
  5. Let sponsors choose and justify comparators7%
  1. Test clinician and AI together66%
  2. Match comparator to intended use49%
  3. Also report device-alone performance29%
  4. Use specialists for specialist tasks28%
All 9 proposals
  1. Do not let average clinicians set bar23%
  2. Report panel agreement and makeup12%
  3. Compare against care without the device9%
  4. Safety minimums that cannot be offset8%
  5. Let sponsors choose and justify comparators0%
  1. Test clinician and AI together55%
  2. Match comparator to intended use25%
  3. Safety minimums that cannot be offset20%
  4. Also report device-alone performance15%
All 9 proposals
  1. Use specialists for specialist tasks15%
  2. Do not let average clinicians set bar15%
  3. Compare against care without the device10%
  4. Report panel agreement and makeup5%
  5. Let sponsors choose and justify comparators0%

Only 1 filer in this group, so counts are shown, not shares.

  1. Test clinician and AI together1
  2. Match comparator to intended use1
  3. Let sponsors choose and justify comparators1
  4. Also report device-alone performance0
All 9 proposals
  1. Use specialists for specialist tasks0
  2. Compare against care without the device0
  3. Do not let average clinicians set bar0
  4. Report panel agreement and makeup0
  5. Safety minimums that cannot be offset0
  1. Test clinician and AI together61%
  2. Match comparator to intended use42%
  3. Also report device-alone performance26%
  4. Use specialists for specialist tasks16%
All 9 proposals
  1. Compare against care without the device16%
  2. Report panel agreement and makeup10%
  3. Safety minimums that cannot be offset10%
  4. Do not let average clinicians set bar3%
  5. Let sponsors choose and justify comparators3%

What should the default comparator be?

36%26%19%9%9%
41%25%19%9%
28%28%17%8%19%

Only 11 filers who took a side in this group, so counts are shown, not shares.

5141

No one in this group answered this part.

Only 14 filers who took a side in this group, so counts are shown, not shares.

3622
Tiered or severalMatch the real workflowCare without the deviceSpecialist panelEvidence-based standardMedian clinician
Question 15 of 26 · Performance against usual care160 filers answered · 32% of all

Could the AI be measured against what would have happened without it: unaided judgment, a delayed specialist, or no intervention?

The FDA’s full question

Are there ways in which performance might be assessed relative to the care, technology, or course of action likely to occur in the absence of the device, rather than to the comparators described in this section? What approaches might be used to identify and justify a comparator such as unaided clinical judgment, delayed specialist review, or no intervention?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. Care without the device is a fair comparison.If backed by evidence from the setting where it will be used.
  2. A weak comparison cannot lower the safety bar.Safety limits hold whatever the comparison shows, filers say.
  3. Clinicians push for tighter oversight here.52% of clinicians' asks, against 35% of industry's.
How these are counted

Asks chart: of the 151 specific asks linked to Q15 (more oversight 63, mixed 3, less oversight 8, neither: method or clarity 77). Proposals: of the 160 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q15 →

What filers proposed

  1. Compare with care likely without device84%
  2. Justify comparator with setting evidence29%
  3. Keep safety floor whatever the comparator17%
  4. Test clinicians with and without device14%
All 9 proposals
  1. Use separate comparators for safety and benefit11%
  2. Build comparator with real-world or trial designs11%
  3. Measure access, timing and patient outcomes9%
  4. Keep standard of care as comparator4%
  5. Leave comparator choice flexible4%
  1. Compare with care likely without device84%
  2. Justify comparator with setting evidence33%
  3. Keep safety floor whatever the comparator14%
  4. Test clinicians with and without device14%
All 9 proposals
  1. Use separate comparators for safety and benefit11%
  2. Build comparator with real-world or trial designs10%
  3. Measure access, timing and patient outcomes7%
  4. Leave comparator choice flexible5%
  5. Keep standard of care as comparator2%
  1. Compare with care likely without device79%
  2. Justify comparator with setting evidence26%
  3. Keep safety floor whatever the comparator21%
  4. Test clinicians with and without device19%
All 9 proposals
  1. Measure access, timing and patient outcomes14%
  2. Use separate comparators for safety and benefit12%
  3. Keep standard of care as comparator12%
  4. Build comparator with real-world or trial designs7%
  5. Leave comparator choice flexible0%

Only 14 filers in this group, so counts are shown, not shares.

  1. Compare with care likely without device10
  2. Build comparator with real-world or trial designs5
  3. Test clinicians with and without device3
  4. Keep safety floor whatever the comparator2
All 9 proposals
  1. Use separate comparators for safety and benefit2
  2. Measure access, timing and patient outcomes2
  3. Justify comparator with setting evidence1
  4. Keep standard of care as comparator0
  5. Leave comparator choice flexible0

Only 2 filers in this group, so counts are shown, not shares.

  1. Compare with care likely without device2
  2. Justify comparator with setting evidence1
  3. Keep safety floor whatever the comparator1
  4. Leave comparator choice flexible1
All 9 proposals
  1. Test clinicians with and without device0
  2. Use separate comparators for safety and benefit0
  3. Build comparator with real-world or trial designs0
  4. Measure access, timing and patient outcomes0
  5. Keep standard of care as comparator0
  1. Compare with care likely without device100%
  2. Justify comparator with setting evidence29%
  3. Keep safety floor whatever the comparator19%
  4. Use separate comparators for safety and benefit10%
All 9 proposals
  1. Test clinicians with and without device5%
  2. Build comparator with real-world or trial designs5%
  3. Measure access, timing and patient outcomes5%
  4. Leave comparator choice flexible5%
  5. Keep standard of care as comparator0%

Which way do the asks push?

42%51%
35%54%
52%45%

Only 12 asks in this group, so counts are shown, not shares.

93

Only 2 asks in this group, so counts are shown, not shares.

2
39%57%
More oversightMixedLess oversightNeither: method or clarity
Question 16 of 26 · Independent third parties209 filers answered · 41% of all

What role should independent third parties play?

The FDA’s full question

Should independent third parties be involved in some or all aspects of a competency-based approach, including device benchmarking and clinical confirmation? If so, in what ways might qualified, independent third-party participation contribute to this approach, and what qualifications and independence criteria should apply? Are there aspects of the assessment for which third-party involvement would be impractical or inadvisable? What safeguards or program design features would be critical to prevent third-party participation from limiting competition or preventing innovation?

In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes

Key findingsFindings describe all filers.

  1. The debate is how, not whether.Filers set terms for evaluators more than they take a side.
  2. Evaluators must be qualified and independent.With several providers, so no single body becomes a gatekeeper.
  3. Industry more often wants them optional.Optional, not a gate: industry 32%, clinicians 12%.
How these are counted

Answer chart: of the 209 filers who answered (accept, with changes 65, accept, as written 3, split 3, reject 3, no verdict 135). Proposals: of the 209 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q16 →

Should independent third parties take part?

31%65%
30%62%
45%55%

Only 15 filers in this group, so counts are shown, not shares.

213

Only 1 filer in this group, so counts are shown, not shares.

1
26%74%
Accept, with changesAccept, as writtenSplitRejectAnswered, no verdict

Of the 74 who gave a verdict, 65 accepted with changes, 3 accepted as written, 3 were split and 3 rejected.

Of the 44 who gave a verdict, 35 accepted with changes, 3 accepted as written, 3 were split and 3 rejected.

All 19 who gave a verdict accepted with changes.

Both who gave a verdict accepted with changes.

No one in this group gave a yes or no.

All 9 who gave a verdict accepted with changes.

What filers proposed

  1. Set qualification and independence rules51%
  2. Allow multiple providers, no single gatekeeper27%
  3. Third parties hold hidden test sets26%
  4. Keep third parties optional, not a gate26%
All 9 proposals
  1. Build on existing accreditation programs23%
  2. Scale third-party role to device risk20%
  3. Keep costs fair for small developers19%
  4. FDA and makers keep final responsibility15%
  5. Use third parties after market release too9%
  1. Set qualification and independence rules51%
  2. Keep third parties optional, not a gate32%
  3. Allow multiple providers, no single gatekeeper28%
  4. Third parties hold hidden test sets25%
All 9 proposals
  1. Build on existing accreditation programs24%
  2. Keep costs fair for small developers22%
  3. Scale third-party role to device risk20%
  4. FDA and makers keep final responsibility14%
  5. Use third parties after market release too6%
  1. Set qualification and independence rules52%
  2. Allow multiple providers, no single gatekeeper33%
  3. Third parties hold hidden test sets26%
  4. Scale third-party role to device risk19%
All 9 proposals
  1. Build on existing accreditation programs17%
  2. Use third parties after market release too17%
  3. Keep costs fair for small developers14%
  4. Keep third parties optional, not a gate12%
  5. FDA and makers keep final responsibility12%

Only 15 filers in this group, so counts are shown, not shares.

  1. Set qualification and independence rules10
  2. Third parties hold hidden test sets7
  3. FDA and makers keep final responsibility4
  4. Build on existing accreditation programs3
All 9 proposals
  1. Scale third-party role to device risk3
  2. Allow multiple providers, no single gatekeeper2
  3. Keep costs fair for small developers2
  4. Keep third parties optional, not a gate1
  5. Use third parties after market release too1

Only 1 filer in this group, so counts are shown, not shares.

  1. Set qualification and independence rules1
  2. Keep third parties optional, not a gate1
  3. Keep costs fair for small developers1
  4. Allow multiple providers, no single gatekeeper0
All 9 proposals
  1. Third parties hold hidden test sets0
  2. Build on existing accreditation programs0
  3. Scale third-party role to device risk0
  4. FDA and makers keep final responsibility0
  5. Use third parties after market release too0
  1. Set qualification and independence rules38%
  2. Build on existing accreditation programs29%
  3. Keep third parties optional, not a gate26%
  4. Third parties hold hidden test sets24%
All 9 proposals
  1. Scale third-party role to device risk24%
  2. Allow multiple providers, no single gatekeeper21%
  3. FDA and makers keep final responsibility18%
  4. Keep costs fair for small developers15%
  5. Use third parties after market release too9%
Question 17 of 26 · Other model architectures85 filers answered · 17% of all

Does the approach still work for devices built on other model architectures, such as multimodal vision-language models and world models?

The FDA’s full question

CDRH is exploring whether a competency-based approach could be applied to devices that incorporate a variety of underlying model architectures, such as multimodal vision-language models and generative or predictive world models. Are there aspects of the described approach that might be ineffective or inapplicable to these devices? What additional or distinct considerations might need to be addressed?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. The framework carries over; the tests change.Same structure, with tests fitted to each kind of model.
  2. The device is judged as built, not the model.Including missing, removed or conflicting inputs.
  3. Industry stresses one framework for all.Named by 15 of 48 industry filers and 2 of 16 clinicians.
How these are counted

Asks chart: of the 113 specific asks linked to Q17 (more oversight 59, mixed 2, less oversight 2, neither: method or clarity 50). Proposals: of the 85 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q17 →

What filers proposed

  1. One framework for all model types25%
  2. Evaluate the complete deployed device19%
  3. Test missing, removed or conflicting inputs18%
  4. Check image findings, omissions and quality14%
All 9 proposals
  1. Publish model-specific supplements or examples11%
  2. Check predictions against real outcomes9%
  3. Test parts separately and together9%
  4. Closed-loop testing for devices that act4%
  5. More clinical evidence where benchmarks are weak2%
  1. One framework for all model types31%
  2. Check image findings, omissions and quality17%
  3. Evaluate the complete deployed device15%
  4. Test missing, removed or conflicting inputs15%
All 9 proposals
  1. Publish model-specific supplements or examples12%
  2. Test parts separately and together10%
  3. Check predictions against real outcomes4%
  4. More clinical evidence where benchmarks are weak4%
  5. Closed-loop testing for devices that act0%

Only 16 filers in this group, so counts are shown, not shares.

  1. Evaluate the complete deployed device5
  2. Check image findings, omissions and quality3
  3. Closed-loop testing for devices that act3
  4. One framework for all model types2
All 9 proposals
  1. Check predictions against real outcomes2
  2. Test missing, removed or conflicting inputs1
  3. Publish model-specific supplements or examples1
  4. Test parts separately and together1
  5. More clinical evidence where benchmarks are weak0

Only 6 filers in this group, so counts are shown, not shares.

  1. Test missing, removed or conflicting inputs2
  2. Check predictions against real outcomes1
  3. Test parts separately and together1
  4. One framework for all model types0
All 9 proposals
  1. Evaluate the complete deployed device0
  2. Check image findings, omissions and quality0
  3. Publish model-specific supplements or examples0
  4. Closed-loop testing for devices that act0
  5. More clinical evidence where benchmarks are weak0

Only 1 filer in this group, so counts are shown, not shares.

  1. One framework for all model types1
  2. Evaluate the complete deployed device1
  3. Test missing, removed or conflicting inputs0
  4. Check image findings, omissions and quality0
All 9 proposals
  1. Publish model-specific supplements or examples0
  2. Check predictions against real outcomes0
  3. Test parts separately and together0
  4. Closed-loop testing for devices that act0
  5. More clinical evidence where benchmarks are weak0

Only 14 filers in this group, so counts are shown, not shares.

  1. Test missing, removed or conflicting inputs5
  2. One framework for all model types3
  3. Evaluate the complete deployed device3
  4. Check predictions against real outcomes3
All 9 proposals
  1. Publish model-specific supplements or examples2
  2. Check image findings, omissions and quality1
  3. Test parts separately and together1
  4. Closed-loop testing for devices that act0
  5. More clinical evidence where benchmarks are weak0

Which way do the asks push?

52%44%
43%52%

Only 13 asks in this group, so counts are shown, not shares.

76

Only 12 asks in this group, so counts are shown, not shares.

111

Only 2 asks in this group, so counts are shown, not shares.

2
62%33%
More oversightMixedLess oversightNeither: method or clarity
Question 18 of 26 · Trading premarket certainty for postmarket monitoring243 filers answered · 48% of all

Can greater premarket uncertainty about a GenAI device’s benefit-risk profile be accepted through greater reliance on postmarket monitoring?

The FDA’s full question

CDRH is considering whether it may be appropriate to accept greater premarket uncertainty regarding a GenAI-enabled device’s benefit-risk profile through greater reliance on postmarket monitoring. Under what conditions might such an approach be appropriate, and what characteristics of a monitoring program would need to be in place to justify reduced premarket evidence? Are there device types or risk profiles for which this approach would not be appropriate?

In the summary report: Takeaway 6, Closer monitoring in place of some premarket testing got a conditional yes

Key findingsFindings describe all filers.

  1. Monitoring can help, but never as a free pass.Almost every yes came with conditions.
  2. High-risk uses stay tested before launch.Plus a plan fixed up front and proof it catches failures.
  3. Industry stresses setting the plan first.A set plan and triggers: industry 41%, clinicians 23%.
How these are counted

Answer chart: of the 243 filers who answered (yes, with conditions 187, yes, broadly 3, no, it only verifies 15, no side 38). Proposals: of the 243 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q18 →

Can postmarket monitoring offset premarket uncertainty?

77%16%
84%11%
69%10%21%
64%32%

Only 4 filers in this group, so counts are shown, not shares.

22
74%17%
Yes, with conditionsYes, broadlyNo, it only verifiesNo side

What filers proposed

  1. Keep high-risk uses tested before launch49%
  2. Set monitoring plan and triggers upfront33%
  3. Prove monitoring catches failures in time33%
  4. Require ability to pause or roll back19%
All 9 proposals
  1. Independent review of monitoring results14%
  2. Log outputs with model version9%
  3. Make monitoring commitments legally enforceable8%
  4. Start with limited, staged rollout8%
  5. Report monitoring results openly4%
  1. Keep high-risk uses tested before launch46%
  2. Set monitoring plan and triggers upfront41%
  3. Prove monitoring catches failures in time33%
  4. Require ability to pause or roll back22%
All 9 proposals
  1. Independent review of monitoring results15%
  2. Make monitoring commitments legally enforceable12%
  3. Start with limited, staged rollout11%
  4. Log outputs with model version8%
  5. Report monitoring results openly5%
  1. Keep high-risk uses tested before launch65%
  2. Prove monitoring catches failures in time31%
  3. Set monitoring plan and triggers upfront23%
  4. Require ability to pause or roll back13%
All 9 proposals
  1. Independent review of monitoring results12%
  2. Log outputs with model version10%
  3. Start with limited, staged rollout6%
  4. Make monitoring commitments legally enforceable4%
  5. Report monitoring results openly4%
  1. Keep high-risk uses tested before launch41%
  2. Prove monitoring catches failures in time36%
  3. Set monitoring plan and triggers upfront27%
  4. Require ability to pause or roll back27%
All 9 proposals
  1. Independent review of monitoring results18%
  2. Log outputs with model version14%
  3. Make monitoring commitments legally enforceable9%
  4. Report monitoring results openly5%
  5. Start with limited, staged rollout0%

Only 4 filers in this group, so counts are shown, not shares.

  1. Keep high-risk uses tested before launch1
  2. Set monitoring plan and triggers upfront1
  3. Log outputs with model version1
  4. Prove monitoring catches failures in time0
All 9 proposals
  1. Require ability to pause or roll back0
  2. Independent review of monitoring results0
  3. Make monitoring commitments legally enforceable0
  4. Start with limited, staged rollout0
  5. Report monitoring results openly0
  1. Keep high-risk uses tested before launch46%
  2. Prove monitoring catches failures in time40%
  3. Set monitoring plan and triggers upfront26%
  4. Require ability to pause or roll back14%
All 9 proposals
  1. Independent review of monitoring results11%
  2. Log outputs with model version11%
  3. Start with limited, staged rollout6%
  4. Report monitoring results openly3%
  5. Make monitoring commitments legally enforceable0%
Question 19 of 26 · Postmarket performance evaluation357 filers answered · 71% of all

How should an AI device be monitored after launch, and what sets the cadence?

The FDA’s full question

Please comment on the potential approaches to postmarket performance evaluation, including periodic re-benchmarking, sample-based clinician review, and performance degradation monitoring. What additional approaches should CDRH consider, and how should the cadence and triggering events for reassessment be determined?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. Model changes should trigger a re-test.Including upstream changes the device maker did not start.
  2. Filers split on a fixed schedule.Some want a calendar floor; others want event triggers only.
  3. Clinicians want near misses tracked.Near misses and overrides: clinicians 34%, industry 13%.
How these are counted

Asks chart: of the 678 specific asks linked to Q19 (more oversight 472, mixed 11, less oversight 18, neither: method or clarity 177). Proposals: of the 357 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q19 →

What filers proposed

  1. Re-test when the model changes32%
  2. More monitoring for higher risk20%
  3. Clinicians review real-world samples20%
  4. Track near misses and overrides20%
All 9 proposals
  1. A schedule plus event triggers16%
  2. Preset pause or rollback thresholds16%
  3. Version-linked records11%
  4. Link to patient outcomes8%
  5. Let sponsors choose methods5%
  1. Re-test when the model changes32%
  2. More monitoring for higher risk23%
  3. A schedule plus event triggers19%
  4. Clinicians review real-world samples17%
All 9 proposals
  1. Track near misses and overrides13%
  2. Preset pause or rollback thresholds13%
  3. Let sponsors choose methods9%
  4. Version-linked records8%
  5. Link to patient outcomes6%
  1. Track near misses and overrides34%
  2. Re-test when the model changes32%
  3. Clinicians review real-world samples29%
  4. More monitoring for higher risk24%
All 9 proposals
  1. Preset pause or rollback thresholds21%
  2. A schedule plus event triggers18%
  3. Link to patient outcomes12%
  4. Version-linked records9%
  5. Let sponsors choose methods0%
  1. Re-test when the model changes35%
  2. Clinicians review real-world samples23%
  3. Track near misses and overrides19%
  4. Preset pause or rollback thresholds19%
All 9 proposals
  1. Version-linked records19%
  2. Link to patient outcomes13%
  3. A schedule plus event triggers6%
  4. More monitoring for higher risk3%
  5. Let sponsors choose methods3%

Only 6 filers in this group, so counts are shown, not shares.

  1. Track near misses and overrides2
  2. More monitoring for higher risk1
  3. Clinicians review real-world samples1
  4. Preset pause or rollback thresholds1
All 9 proposals
  1. Re-test when the model changes0
  2. A schedule plus event triggers0
  3. Version-linked records0
  4. Link to patient outcomes0
  5. Let sponsors choose methods0
  1. Re-test when the model changes31%
  2. Track near misses and overrides21%
  3. Version-linked records21%
  4. More monitoring for higher risk17%
All 9 proposals
  1. Clinicians review real-world samples16%
  2. Preset pause or rollback thresholds16%
  3. A schedule plus event triggers12%
  4. Link to patient outcomes7%
  5. Let sponsors choose methods2%

Which way do the asks push?

70%26%
64%29%
80%19%
72%27%

Only 17 asks in this group, so counts are shown, not shares.

134
73%27%
More oversightMixedLess oversightNeither: method or clarity
Question 20 of 26 · Machine-based supervisory agents201 filers answered · 40% of all

Could AI supervisory agents help carry out postmarket monitoring?

The FDA’s full question

Please comment on whether the proposed approaches to postmarket monitoring can be facilitated by machine-based supervisory agents. What considerations, including the evaluation and reliability of the supervisory agent itself, should CDRH take into account for such an approach?

In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes

Key findingsFindings describe all filers.

  1. Filers are open to it, on conditions.Nearly all who took a side add conditions.
  2. The supervisor needs its own testing.And it should be independent of the device it watches.
  3. Clinicians want humans to have the final say.Asked for by 51% of clinicians and 33% of industry.
How these are counted

Answer chart: of the 201 filers who answered (accept, with changes 67, accept, as written 2, split 1, reject 0, no verdict 131). Proposals: of the 201 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q20 →

Could supervisory agents help with monitoring?

33%65%
39%59%
30%70%

Only 15 filers in this group, so counts are shown, not shares.

213

Only 1 filer in this group, so counts are shown, not shares.

1
24%72%
Accept, with changesAccept, as writtenSplitAnswered, no verdict

Of the 70 who gave a verdict, 67 accepted with changes, 2 accepted as written and 1 was split.

Of the 50 who gave a verdict, 48 accepted with changes, 1 accepted as written and 1 was split.

All 11 who gave a verdict accepted with changes.

Both who gave a verdict accepted with changes.

No one in this group gave a yes or no.

Of the 7 who gave a verdict, 6 accepted with changes and 1 accepted as written.

What filers proposed

  1. Test the supervisor's own accuracy77%
  2. Keep supervisor independent from the device50%
  3. Humans make the final safety call35%
  4. Version and re-check supervisor after changes22%
All 9 proposals
  1. Humans spot-check flagged and unflagged cases16%
  2. Keep audit logs of supervisor decisions11%
  3. Use fixed rule checks alongside AI8%
  4. Limit what the supervisor can do8%
  5. Fail safe when supervisor fails7%
  1. Test the supervisor's own accuracy79%
  2. Keep supervisor independent from the device52%
  3. Humans make the final safety call33%
  4. Version and re-check supervisor after changes20%
All 9 proposals
  1. Humans spot-check flagged and unflagged cases15%
  2. Keep audit logs of supervisor decisions13%
  3. Limit what the supervisor can do11%
  4. Use fixed rule checks alongside AI11%
  5. Fail safe when supervisor fails7%
  1. Test the supervisor's own accuracy70%
  2. Humans make the final safety call51%
  3. Keep supervisor independent from the device49%
  4. Version and re-check supervisor after changes19%
All 9 proposals
  1. Humans spot-check flagged and unflagged cases14%
  2. Keep audit logs of supervisor decisions5%
  3. Use fixed rule checks alongside AI3%
  4. Limit what the supervisor can do0%
  5. Fail safe when supervisor fails0%

Only 15 filers in this group, so counts are shown, not shares.

  1. Test the supervisor's own accuracy14
  2. Humans spot-check flagged and unflagged cases5
  3. Keep supervisor independent from the device4
  4. Humans make the final safety call4
All 9 proposals
  1. Version and re-check supervisor after changes4
  2. Use fixed rule checks alongside AI2
  3. Keep audit logs of supervisor decisions0
  4. Limit what the supervisor can do0
  5. Fail safe when supervisor fails0

Only 1 filer in this group, so counts are shown, not shares.

  1. Test the supervisor's own accuracy1
  2. Keep supervisor independent from the device0
  3. Humans make the final safety call0
  4. Version and re-check supervisor after changes0
All 9 proposals
  1. Humans spot-check flagged and unflagged cases0
  2. Keep audit logs of supervisor decisions0
  3. Use fixed rule checks alongside AI0
  4. Limit what the supervisor can do0
  5. Fail safe when supervisor fails0
  1. Test the supervisor's own accuracy68%
  2. Keep supervisor independent from the device60%
  3. Version and re-check supervisor after changes32%
  4. Humans make the final safety call28%
All 9 proposals
  1. Fail safe when supervisor fails20%
  2. Keep audit logs of supervisor decisions16%
  3. Humans spot-check flagged and unflagged cases12%
  4. Limit what the supervisor can do8%
  5. Use fixed rule checks alongside AI4%
Question 21 of 26 · Clinicians, institutions and societies251 filers answered · 50% of all

What roles should clinicians and institutions play in monitoring, without diluting manufacturer accountability?

The FDA’s full question

What roles might clinicians, healthcare institutions, professional societies, standards-setting bodies, and other stakeholders appropriately play in postmarket monitoring, and how can these roles be structured without diffusing manufacturer accountability?

In the summary report: Takeaway 7, The device maker stays responsible, a majority said, even for AI it did not build

Key findingsFindings describe all filers.

  1. Others can help; the maker stays accountable.Hospitals and clinicians report signals; the maker must act.
  2. Each party's role should be written down.In plans or labeling, without moving the maker's duties.
  3. Clinicians more often oppose new duties.Opposing added burden: clinicians 33%, industry 9%.
How these are counted

Answer chart: of the 208 filers who took a side on this point (manufacturer first 180, whoever controls the change 28). The answer chart counts only filers who named one option; proposal shares are of all who answered. Proposals: of the 251 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q21 →

What filers proposed

  1. Manufacturer stays accountable for the device80%
  2. Write down who does what35%
  3. Societies and standards bodies set methods18%
  4. Do not burden hospitals and clinicians17%
All 9 proposals
  1. Easy reporting channels for frontline users15%
  2. Independent evaluators and shared monitoring networks10%
  3. Manufacturers share logs and change notices9%
  4. Build on existing reporting systems5%
  5. Let sites pause or restrict use3%
  1. Manufacturer stays accountable for the device80%
  2. Write down who does what37%
  3. Societies and standards bodies set methods16%
  4. Do not burden hospitals and clinicians9%
All 9 proposals
  1. Easy reporting channels for frontline users9%
  2. Independent evaluators and shared monitoring networks7%
  3. Manufacturers share logs and change notices5%
  4. Build on existing reporting systems3%
  5. Let sites pause or restrict use2%
  1. Manufacturer stays accountable for the device85%
  2. Write down who does what35%
  3. Do not burden hospitals and clinicians33%
  4. Easy reporting channels for frontline users25%
All 9 proposals
  1. Societies and standards bodies set methods18%
  2. Manufacturers share logs and change notices15%
  3. Independent evaluators and shared monitoring networks7%
  4. Build on existing reporting systems6%
  5. Let sites pause or restrict use4%
  1. Manufacturer stays accountable for the device71%
  2. Write down who does what38%
  3. Independent evaluators and shared monitoring networks29%
  4. Societies and standards bodies set methods24%
All 9 proposals
  1. Easy reporting channels for frontline users14%
  2. Do not burden hospitals and clinicians10%
  3. Manufacturers share logs and change notices5%
  4. Let sites pause or restrict use5%
  5. Build on existing reporting systems0%

Only 2 filers in this group, so counts are shown, not shares.

  1. Manufacturer stays accountable for the device2
  2. Easy reporting channels for frontline users1
  3. Write down who does what0
  4. Societies and standards bodies set methods0
All 9 proposals
  1. Do not burden hospitals and clinicians0
  2. Independent evaluators and shared monitoring networks0
  3. Manufacturers share logs and change notices0
  4. Build on existing reporting systems0
  5. Let sites pause or restrict use0
  1. Manufacturer stays accountable for the device75%
  2. Write down who does what32%
  3. Societies and standards bodies set methods20%
  4. Do not burden hospitals and clinicians15%
All 9 proposals
  1. Easy reporting channels for frontline users12%
  2. Independent evaluators and shared monitoring networks12%
  3. Manufacturers share logs and change notices12%
  4. Build on existing reporting systems10%
  5. Let sites pause or restrict use2%

Who carries postmarket accountability?

87%13%
85%15%
92%8%

Only 15 filers who took a side in this group, so counts are shown, not shares.

132

Only 2 filers who took a side in this group, so counts are shown, not shares.

2
82%18%
Manufacturer firstWhoever controls the change
Question 22 of 26 · Re-benchmarking after a modification231 filers answered · 46% of all

With the premarket competency assessment as the baseline, which post-deployment changes need re-evaluation, and how much?

The FDA’s full question

CDRH envisions that a manufacturer’s premarket competency-based assessment might serve as a baseline against which post-deployment modifications could be re-evaluated. How might the extent of re-benchmarking or other evidence be scaled to the nature and expected impact of a given modification? For example, are there categories of change that might not significantly affect safety or effectiveness of a GenAI-enabled device, or might be appropriately managed within a sponsor’s quality management system versus requiring FDA premarket review and authorization before implementation? Are there categories of changes that might be appropriate for inclusion in a PCCP?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. Clinical effect sets how much to re-test.Not how much code or configuration changed.
  2. Small changes in-house, big ones back to FDA.New uses, patients or autonomy need a new FDA review.
  3. Filers split on new model versions.Some allow them in a change plan; others want FDA review.
How these are counted

Asks chart: of the 439 specific asks linked to Q22 (more oversight 170, mixed 24, less oversight 53, neither: method or clarity 192). Proposals: of the 231 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q22 →

What filers proposed

  1. Scale re-testing to clinical impact50%
  2. Cover bounded changes in change plans39%
  3. Handle low-impact changes in-house33%
  4. Require FDA review for major changes21%
All 9 proposals
  1. FDA should publish examples and criteria12%
  2. Re-test whenever model or setup changes11%
  3. Tie evidence to a versioned setup10%
  4. Always re-run a fixed safety test set5%
  5. Keep existing FDA change-control rules3%
  1. Scale re-testing to clinical impact49%
  2. Cover bounded changes in change plans41%
  3. Handle low-impact changes in-house37%
  4. Require FDA review for major changes22%
All 9 proposals
  1. FDA should publish examples and criteria13%
  2. Tie evidence to a versioned setup11%
  3. Re-test whenever model or setup changes9%
  4. Always re-run a fixed safety test set6%
  5. Keep existing FDA change-control rules5%
  1. Scale re-testing to clinical impact52%
  2. Cover bounded changes in change plans35%
  3. Handle low-impact changes in-house35%
  4. Require FDA review for major changes25%
All 9 proposals
  1. Re-test whenever model or setup changes20%
  2. FDA should publish examples and criteria8%
  3. Always re-run a fixed safety test set5%
  4. Tie evidence to a versioned setup2%
  5. Keep existing FDA change-control rules0%

Only 19 filers in this group, so counts are shown, not shares.

  1. Scale re-testing to clinical impact12
  2. Cover bounded changes in change plans5
  3. Handle low-impact changes in-house5
  4. Tie evidence to a versioned setup3
All 9 proposals
  1. Require FDA review for major changes2
  2. Re-test whenever model or setup changes2
  3. FDA should publish examples and criteria1
  4. Always re-run a fixed safety test set0
  5. Keep existing FDA change-control rules0

Only 3 filers in this group, so counts are shown, not shares.

  1. Require FDA review for major changes2
  2. Cover bounded changes in change plans1
  3. Scale re-testing to clinical impact0
  4. Handle low-impact changes in-house0
All 9 proposals
  1. FDA should publish examples and criteria0
  2. Re-test whenever model or setup changes0
  3. Tie evidence to a versioned setup0
  4. Always re-run a fixed safety test set0
  5. Keep existing FDA change-control rules0
  1. Scale re-testing to clinical impact49%
  2. Cover bounded changes in change plans43%
  3. Handle low-impact changes in-house26%
  4. FDA should publish examples and criteria17%
All 9 proposals
  1. Require FDA review for major changes14%
  2. Re-test whenever model or setup changes9%
  3. Tie evidence to a versioned setup9%
  4. Always re-run a fixed safety test set3%
  5. Keep existing FDA change-control rules0%

Which way do the asks push?

39%12%44%
34%14%47%
44%10%38%
59%34%

Only 3 asks in this group, so counts are shown, not shares.

3
44%14%41%
More oversightMixedLess oversightNeither: method or clarity
Question 23 of 26 · Predetermined change control plans (PCCPs)176 filers answered · 35% of all

How can a change-control plan cover changes that cannot be fully specified in advance?

The FDA’s full question

How might PCCP concepts or other change-control approaches be adapted for GenAI-enabled devices when the nature or scope of future modifications cannot be fully prespecified?

In the summary report: Takeaway 3, More than half asked the FDA for more oversight than it proposed

Key findingsFindings describe all filers.

  1. Plans can fix the test, not each change.Bounded types of change, each with pass rules set in advance.
  2. Plans need a way to undo a change.Rollback and staged release, plus monitoring after each change.
  3. Clinicians stress FDA review beyond the plan.Changes outside it go back to FDA: clinicians 38%, industry 21%.
How these are counted

Asks chart: of the 235 specific asks linked to Q23 (more oversight 71, mixed 24, less oversight 48, neither: method or clarity 92). Proposals: of the 176 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q23 →

What filers proposed

  1. Allow bounded categories of change48%
  2. Prespecify the test, not the change43%
  3. Send out-of-bounds changes back to FDA25%
  4. Require rollback and staged release22%
All 9 proposals
  1. Monitor performance after changes15%
  2. Track versions and notify users14%
  3. FDA should clarify with examples14%
  4. Sort changes into risk or component tiers8%
  5. Keep core safety properties unchanged6%
  1. Allow bounded categories of change45%
  2. Prespecify the test, not the change43%
  3. Send out-of-bounds changes back to FDA21%
  4. Require rollback and staged release21%
All 9 proposals
  1. FDA should clarify with examples16%
  2. Monitor performance after changes15%
  3. Sort changes into risk or component tiers11%
  4. Track versions and notify users9%
  5. Keep core safety properties unchanged5%
  1. Allow bounded categories of change46%
  2. Prespecify the test, not the change42%
  3. Send out-of-bounds changes back to FDA38%
  4. Require rollback and staged release23%
All 9 proposals
  1. Track versions and notify users23%
  2. Monitor performance after changes19%
  3. Keep core safety properties unchanged15%
  4. FDA should clarify with examples4%
  5. Sort changes into risk or component tiers4%

Only 11 filers in this group, so counts are shown, not shares.

  1. Allow bounded categories of change7
  2. Prespecify the test, not the change6
  3. Send out-of-bounds changes back to FDA3
  4. Require rollback and staged release1
All 9 proposals
  1. Track versions and notify users1
  2. FDA should clarify with examples1
  3. Monitor performance after changes0
  4. Sort changes into risk or component tiers0
  5. Keep core safety properties unchanged0

Only 1 filer in this group, so counts are shown, not shares.

  1. Allow bounded categories of change1
  2. Prespecify the test, not the change0
  3. Send out-of-bounds changes back to FDA0
  4. Require rollback and staged release0
All 9 proposals
  1. Monitor performance after changes0
  2. Track versions and notify users0
  3. FDA should clarify with examples0
  4. Sort changes into risk or component tiers0
  5. Keep core safety properties unchanged0
  1. Allow bounded categories of change58%
  2. Prespecify the test, not the change38%
  3. Send out-of-bounds changes back to FDA29%
  4. Require rollback and staged release29%
All 9 proposals
  1. Track versions and notify users29%
  2. Monitor performance after changes21%
  3. FDA should clarify with examples17%
  4. Sort changes into risk or component tiers4%
  5. Keep core safety properties unchanged0%

Which way do the asks push?

30%10%20%39%
27%10%21%42%
39%13%26%23%

Only 10 asks in this group, so counts are shown, not shares.

514

Only 1 asks in this group, so counts are shown, not shares.

1
28%9%19%44%
More oversightMixedLess oversightNeither: method or clarity
Question 24 of 26 · Third-party foundation model changes260 filers answered · 52% of all

When the foundation model’s developer changes the model, how does the device maker detect it and respond, so safety and effectiveness are not compromised?

The FDA’s full question

For devices built on third-party foundation models, changes to the underlying model may be initiated by the third-party model developer rather than the device manufacturer. How can a manufacturer detect, evaluate, and respond to such changes in a timely manner? What mechanisms—for example, contractual, technical, or through a PCCP—could provide reasonable assurance that third-party developer-initiated changes do not compromise the safety or effectiveness of the device?

In the summary report: Takeaway 7, The device maker stays responsible, a majority said, even for AI it did not build

Key findingsFindings describe all filers.

  1. A layered answer: pin, watch, re-test.Locking the version, catching silent changes, re-testing first.
  2. Contract terms divide filers.Some want notice terms required; others say makers can't.
  3. Clinicians stress the maker's responsibility.Keeping the maker responsible: clinicians 23%, industry 11%.
How these are counted

Asks chart: of the 559 specific asks linked to Q24 (more oversight 343, mixed 10, less oversight 30, neither: method or clarity 176). Proposals: of the 260 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q24 →

What filers proposed

  1. Re-test before using a changed model58%
  2. Track the version and detect changes54%
  3. Get advance notice of model changes53%
  4. Pin or lock the model version41%
All 9 proposals
  1. Keep a rollback or safe fallback32%
  2. Cover model updates in a change plan25%
  3. Keep the manufacturer responsible12%
  4. Keep requirements flexible and feasible11%
  5. Tell users and hospitals about changes6%
  1. Re-test before using a changed model55%
  2. Get advance notice of model changes55%
  3. Track the version and detect changes52%
  4. Pin or lock the model version45%
All 9 proposals
  1. Keep a rollback or safe fallback31%
  2. Cover model updates in a change plan29%
  3. Keep requirements flexible and feasible15%
  4. Keep the manufacturer responsible11%
  5. Tell users and hospitals about changes4%
  1. Re-test before using a changed model68%
  2. Get advance notice of model changes53%
  3. Track the version and detect changes51%
  4. Pin or lock the model version40%
All 9 proposals
  1. Keep a rollback or safe fallback38%
  2. Cover model updates in a change plan30%
  3. Keep the manufacturer responsible23%
  4. Tell users and hospitals about changes13%
  5. Keep requirements flexible and feasible6%
  1. Track the version and detect changes75%
  2. Re-test before using a changed model50%
  3. Get advance notice of model changes45%
  4. Pin or lock the model version20%
All 9 proposals
  1. Keep a rollback or safe fallback15%
  2. Cover model updates in a change plan10%
  3. Keep the manufacturer responsible5%
  4. Tell users and hospitals about changes5%
  5. Keep requirements flexible and feasible0%

Only 3 filers in this group, so counts are shown, not shares.

  1. Track the version and detect changes2
  2. Get advance notice of model changes1
  3. Keep a rollback or safe fallback1
  4. Re-test before using a changed model0
All 9 proposals
  1. Pin or lock the model version0
  2. Cover model updates in a change plan0
  3. Keep the manufacturer responsible0
  4. Keep requirements flexible and feasible0
  5. Tell users and hospitals about changes0
  1. Re-test before using a changed model64%
  2. Track the version and detect changes55%
  3. Get advance notice of model changes48%
  4. Pin or lock the model version36%
All 9 proposals
  1. Keep a rollback or safe fallback36%
  2. Cover model updates in a change plan12%
  3. Keep requirements flexible and feasible9%
  4. Tell users and hospitals about changes9%
  5. Keep the manufacturer responsible6%

Which way do the asks push?

61%31%
58%32%
57%33%
83%17%

Only 6 asks in this group, so counts are shown, not shares.

51
65%35%
More oversightMixedLess oversightNeither: method or clarity
Question 25 of 26 · Foundation Model Master Files179 filers answered · 36% of all

Would voluntary Foundation Model Master Files be practical, and useful in premarket review?

The FDA’s full question

Would voluntary Foundation Model MAFs be practical for foundation model developers to provide and to keep current, and what content might they include? Given that participation would be voluntary and that model developers may have limited incentive to disclose safety-relevant information, what would make such a program sufficiently useful for premarket review? Are there alternative mechanisms CDRH should consider for obtaining information about underlying foundation models?

In the summary report: Takeaway 5, The FDA’s core proposals won broad support, but most filers wanted changes

Key findingsFindings describe all filers.

  1. Seen as useful, if the FDA sets the content.Such as data sources, known limits and version history.
  2. A missing file needs a defined route.Filers split on whether its absence should cost the sponsor.
  3. Clinicians ask more for open model details.With users or the public: 28% of clinicians, 3% of industry.
How these are counted

Answer chart: of the 179 filers who answered (accept, with changes 85, accept, as written 12, split 0, reject 2, no verdict 80). Proposals: of the 179 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q25 →

Would voluntary master files be useful?

47%45%
51%9%38%
45%52%

Only 14 filers in this group, so counts are shown, not shares.

77

Only 3 filers in this group, so counts are shown, not shares.

12
37%57%
Accept, with changesAccept, as writtenRejectAnswered, no verdict

Of the 99 who gave a verdict, 85 accepted with changes, 12 accepted as written and 2 rejected.

Of the 64 who gave a verdict, 53 accepted with changes, 9 accepted as written and 2 rejected.

Of the 14 who gave a verdict, 13 accepted with changes and 1 accepted as written.

All 7 who gave a verdict accepted with changes.

The one filer who gave a verdict accepted with changes.

Of the 13 who gave a verdict, 11 accepted with changes and 2 accepted as written.

What filers proposed

  1. Set standard content for model files52%
  2. Commit to notice of model changes40%
  3. Define a route when no file exists26%
  4. Give review credit for using a file12%
All 9 proposals
  1. Use contracts with model providers11%
  2. Make model disclosure mandatory10%
  3. Sponsors record exact model version10%
  4. Use independent third-party verification8%
  5. Share model information with users or public8%
  1. Set standard content for model files50%
  2. Commit to notice of model changes43%
  3. Define a route when no file exists26%
  4. Give review credit for using a file13%
All 9 proposals
  1. Use contracts with model providers10%
  2. Make model disclosure mandatory9%
  3. Use independent third-party verification8%
  4. Sponsors record exact model version7%
  5. Share model information with users or public3%
  1. Set standard content for model files41%
  2. Commit to notice of model changes34%
  3. Define a route when no file exists28%
  4. Share model information with users or public28%
All 9 proposals
  1. Give review credit for using a file17%
  2. Use contracts with model providers17%
  3. Make model disclosure mandatory10%
  4. Use independent third-party verification10%
  5. Sponsors record exact model version7%

Only 14 filers in this group, so counts are shown, not shares.

  1. Set standard content for model files7
  2. Commit to notice of model changes7
  3. Use contracts with model providers3
  4. Share model information with users or public3
All 9 proposals
  1. Define a route when no file exists2
  2. Make model disclosure mandatory2
  3. Sponsors record exact model version2
  4. Give review credit for using a file1
  5. Use independent third-party verification0

Only 3 filers in this group, so counts are shown, not shares.

  1. Make model disclosure mandatory2
  2. Sponsors record exact model version2
  3. Set standard content for model files1
  4. Commit to notice of model changes1
All 9 proposals
  1. Define a route when no file exists0
  2. Give review credit for using a file0
  3. Use contracts with model providers0
  4. Use independent third-party verification0
  5. Share model information with users or public0
  1. Set standard content for model files70%
  2. Define a route when no file exists33%
  3. Commit to notice of model changes30%
  4. Sponsors record exact model version17%
All 9 proposals
  1. Use independent third-party verification13%
  2. Give review credit for using a file7%
  3. Use contracts with model providers7%
  4. Make model disclosure mandatory7%
  5. Share model information with users or public0%
Question 26 of 26 · Agentic devices232 filers answered · 46% of all

What extra oversight does an AI that plans and acts in multiple steps need?

The FDA’s full question

Are there additional considerations that inform the premarket and postmarket evaluation of agentic GenAI-enabled devices, beyond those applicable to non-agentic GenAI-enabled devices? How should the elevated risk associated with autonomous multi-step action, tool use, and reduced opportunity for human review be reflected in acceptance criteria and oversight?

In the summary report: Takeaway 8, How much an AI may do on its own should depend on the risk, filers said

Key findingsFindings describe all filers.

  1. Agents need sign-off, records and limits.And tests of the whole chain of actions, not single steps.
  2. Separate rules for agents divide filers.Some call action-taking high risk by default; others say no.
  3. Industry is likelier to keep current rules.Using existing rules, no new layer: industry 12%, clinicians 2%.
How these are counted

Asks chart: of the 508 specific asks linked to Q26 (more oversight 378, mixed 6, less oversight 11, neither: method or clarity 113). Proposals: of the 232 filers who answered, counted from each filing’s coded summary and asks; a filer can name several. A check of full filings found more proposals than these counts, so each share is a minimum. Groups under 20 show counts. How this was counted

Filings that name Q26 →

What filers proposed

  1. Human sign-off before risky actions38%
  2. Keep full records of agent actions34%
  3. Test the whole chain of actions33%
  4. Limit what the agent may do31%
All 9 proposals
  1. Scale rules to autonomy and consequence28%
  2. Require tested stop and rollback controls24%
  3. Test for manipulation and tool failures17%
  4. Prove human reviewers can really intervene9%
  5. Use existing rules, no new layer8%
  1. Human sign-off before risky actions38%
  2. Test the whole chain of actions36%
  3. Limit what the agent may do29%
  4. Keep full records of agent actions28%
All 9 proposals
  1. Scale rules to autonomy and consequence25%
  2. Require tested stop and rollback controls21%
  3. Test for manipulation and tool failures20%
  4. Use existing rules, no new layer12%
  5. Prove human reviewers can really intervene7%
  1. Human sign-off before risky actions48%
  2. Keep full records of agent actions38%
  3. Test the whole chain of actions38%
  4. Scale rules to autonomy and consequence36%
All 9 proposals
  1. Limit what the agent may do32%
  2. Require tested stop and rollback controls32%
  3. Test for manipulation and tool failures18%
  4. Prove human reviewers can really intervene14%
  5. Use existing rules, no new layer2%
  1. Human sign-off before risky actions40%
  2. Keep full records of agent actions40%
  3. Test the whole chain of actions35%
  4. Scale rules to autonomy and consequence30%
All 9 proposals
  1. Require tested stop and rollback controls10%
  2. Limit what the agent may do5%
  3. Test for manipulation and tool failures5%
  4. Prove human reviewers can really intervene5%
  5. Use existing rules, no new layer0%

Only 1 filer in this group, so counts are shown, not shares.

  1. Human sign-off before risky actions1
  2. Keep full records of agent actions1
  3. Test the whole chain of actions1
  4. Limit what the agent may do1
All 9 proposals
  1. Scale rules to autonomy and consequence1
  2. Require tested stop and rollback controls1
  3. Prove human reviewers can really intervene1
  4. Test for manipulation and tool failures0
  5. Use existing rules, no new layer0
  1. Limit what the agent may do55%
  2. Keep full records of agent actions45%
  3. Require tested stop and rollback controls30%
  4. Human sign-off before risky actions24%
All 9 proposals
  1. Scale rules to autonomy and consequence21%
  2. Test for manipulation and tool failures15%
  3. Test the whole chain of actions9%
  4. Prove human reviewers can really intervene9%
  5. Use existing rules, no new layer9%

Which way do the asks push?

74%22%
73%23%
80%19%
69%31%

Only 3 asks in this group, so counts are shown, not shares.

3
73%23%
More oversightMixedLess oversightNeither: method or clarity