Coalition for State Sandboxes (Krista Drobac, Dr. Kavita Patel)

The Coalition for State Sandboxes (the “Coalition”) appreciates the opportunity to respond to CDRH’s discussion paper. The Coalition is a multi-stakeholder group that supports the creation and responsible operation of state healthcare AI regulatory sandboxes. In that model, participants test AI tools under tailored, time-limited regulatory relief and government-supervised monitoring, a model Utah has put into practice for healthcare AI. Our comments draw on our members’ experience with these programs and focus on Questions 1, 5, 8, 11, 12, 16, 18, 19, 21, 22, 24, and 25.

October 19, 2026
Dockets Management Staff, Food and Drug Administration · Submitted via Regulations.gov
Re: Docket No. FDA-2026-N-7874 — Considerations for the Regulation of Generative AIEnabled Medical Devices: Discussion Paper and Request for Feedback
Dear Dockets Management Staff and the Digital Health Center of Excellence:
The Coalition for State Sandboxes (the “Coalition”) appreciates the opportunity to respond
to CDRH’s discussion paper. The Coalition is a multi-stakeholder group that supports the
creation and responsible operation of state healthcare AI regulatory sandboxes. In that
model, participants test AI tools under tailored, time-limited regulatory relief and
government-supervised monitoring, a model Utah has put into practice for healthcare AI.
Our comments draw on our members’ experience with these programs and focus on
Questions 1, 5, 8, 11, 12, 16, 18, 19, 21, 22, 24, and 25.
Summary of our position
Sandbox programs have become practical evidence engines for the same questions CDRH is
asking, and we offer three points from that experience:
• Supervised pilots fit the competency-based model. A sandbox is a controlled venue
for the shadow-deployment and prospective rungs of clinical confirmation and for
post-market monitoring, and it should be recognized as a source of evidence.
• Post-market reliance (Question 18) works only with independent verification.
Early sandbox experience shows why self-reported performance is not enough.
• FDA should partner explicitly with states on healthcare AI sandboxes. Sandbox
participants need clarity on device status, evidence reuse, and data sharing, and FDA
and state programs need a defined coordination path.
1. The two-axis framework matches how sandboxes already scale oversight
(Questions 1 and 8)
We support the two-axis framework as an organizing heuristic. Sandbox agreements
already work this way: terms are negotiated case by case, and regulators select pilots
where they judge the risk can be mitigated. We recommend two refinements.
First, as Question 1 invites, the grid should credit downstream safeguards and reversibility.
Clinician checkpoints, escalation protocols, enrollment caps, and defined stop rules change
real-world risk, and sandbox participants can document them.
Second, the consequence axis should account for repeated reliance. One early state sandbox
pilot, autonomous prescription refills for chronic conditions, involved a function that looks
low-consequence per transaction. A state medical licensing board nonetheless asked the
sandbox regulator to suspend it, citing the possibility that patients could keep refilling
without clinical reassessment. Risk can compound across encounters even when no single
Coalition for State Sandboxes
1100 G Street, NW, Suite 420, Washington, DC 20005
output is severe, which supports CDRH’s proposal to assess multi-turn trajectories
(Question 5) and extends it to longitudinal use.
2. Greater postmarket reliance needs independent verification (Questions 16, 18, 19,
and 21)
The Coalition supports CDRH’s willingness to consider accepting more premarket
uncertainty in exchange for stronger postmarket monitoring. Sandboxes make that trade
concrete, and early experience shows what makes it credible. Independent verification
here means periodic, structured scrutiny, not continuous third-party gatekeeping:
independent audit on a defined cadence, sample-based clinician review, and rebenchmarking against prespecified thresholds.
After that pilot drew scrutiny, Utah’s sandbox program asked who would verify
performance claims. In October 2026 it announced agreements with six independent
evaluators, drawn from nonprofit, academic, and commercial organizations, to audit
decision accuracy and verify safety and performance claims. It also says it has tightened its
reporting standards since the first agreement.
Generalizing from this and the input of relevant stakeholders, we recommend that FDA:
• Condition reduced premarket evidence on a defined monitoring program.
Minimum elements should be prespecified benchmark thresholds, a re-benchmarking
cadence and triggering events (Question 19), independent sample-based clinician
review, and a stop rule tied to degradation.
• Scale evidence requirements in proportion to risk. Autonomy does not inherently
mean risk, so the trade should not categorically exclude autonomous functions.
Evidence requirements should be proportionate to the risk of each function, while the
highest-consequence functions keep stronger premarket confirmation.
• Define independence for third parties (Question 16). In this model, evaluators are
paid by the participants they assess, which is practical but carries conflict risk. Criteria
should address fee structure, assignment or rotation, and disclosure, extending the
paper’s adjudicator-independence principle.
• Build on ASCA and MDDT while allowing for unique attributes. An FDArecognized evaluator pathway would let sandbox-proven evaluators contribute
without limiting competition.
• Keep accountability with manufacturers (Question 21). Consortia and publicprivate coalitions can convene evaluators and pool results, but should not dilute the
manufacturer’s responsibility.
3. Sandboxes can supply clinical confirmation evidence (Questions 11 and 12)
CDRH describes a ladder of confirmation approaches of rising rigor and patient exposure,
and says a prospective study may not be needed in every case. A sandbox offers a
structured way to climb that ladder: defined populations, volume limits, disclosure to

Coalition for State Sandboxes
1100 G Street, NW, Suite 420, Washington, DC 20005
patients, clinician oversight, and regulator-visible monitoring, all agreed before launch. We
recommend that FDA:
• State that it will consider sandbox-generated evidence when the pilot follows a
prespecified protocol, uses adjudicators independent of the sponsor and any
foundation model developer, and gives FDA access to the underlying results.
• Address how pilots relate to human-subjects and investigational device requirements.
The paper notes that 21 CFR Parts 50, 56, and 812 may apply but leaves them out of
scope; sponsors in sandboxes need that answer.
• Allow benchmarking and real-world pilot evidence to be combined (Question 12) into
one performance estimate when the method is prespecified and the differences
between the two data sources are disclosed.
4. Third-party model changes and verifiable disclosure (Questions 22, 24, and 25)
Question 25 notes that foundation model developers have limited incentive to disclose
safety-relevant information. Sandboxes offer a practical lever: participation can be
conditioned on the commitments CDRH lists in its Master File footnote, such as updatenotification commitments and audit-log availability. We suggest that FDA:
• Treat material foundation model version changes as a re-benchmarking trigger,
with the scope of re-evaluation proportionate to the nature of the change
(Questions 22 and 24). A material change is one that can alter the model’s behavior
in ways relevant to the device’s intended use, such as a version upgrade that changes
output distributions, performance characteristics, or functional capabilities. Routine
patches and minor updates that do not affect performance for the claimed use would
not trigger re-benchmarking. Sandbox participants can pin model versions by contract
and re-run the baseline benchmark before adopting a material update.
• Consider verifiable attestations as a complement to Master Files (Question 25).
Voluntary technical standards such as OVERT, which one state sandbox now requires
of designated vendors and systems, produce independently verifiable “receipts”. Per
the standard’s developer, these receipts evidence clinical oversight, evaluation, and
audit without disclosing model weights, raw prompts or outputs, or personal data.
They do not certify safety, so they supplement rather than replace FDA’s review, but
they address the confidentiality concerns that deter disclosure.
5. Federal-state coordination (Question 21 and Section VI.B)
The paper names state authorities and public-private consortia among the stakeholders
with roles in monitoring, but it does not say how those roles connect to the role of the FDA.
State relief does not change the Food, Drug and Cosmetics Act, and sandbox participants
need to know where they stand. We ask that CDRH:
• Partner explicitly with states on healthcare AI sandboxes, recognizing state
sandbox programs as standing counterparts in the testing and monitoring of AIenabled care models.
Coalition for State Sandboxes
1100 G Street, NW, Suite 420, Washington, DC 20005
• Establish a Pre-Submission-style process for sandbox participants to get timely
feedback on how FDA views a given function, including whether it is a device function,
falls under enforcement discretion, or is excluded under section 520(o). Pilots now
include autonomous action-taking such as prescription renewal, and sponsors need a
predictable channel for that feedback rather than a formal classification
determination.
• Designate a Digital Health Center of Excellence point of contact for state sandbox
programs, as the FDA-facing entry point for that Pre-Submission-style channel when a
pilot design touches device functions.
• Set data-sharing expectations so that evaluator findings can reach FDA with
appropriate confidentiality.
• Consider additional ways to share data to improve evaluation, assessment,
benchmarking and other key areas of interest.
The Coalition is glad to share de-identified lessons learned and to participate in a listening
session. Thank you for considering these comments.
Respectfully submitted,

Krista Drobac Dr. Kavita Patel
Coalition for State Sandboxes Coalition for State Sandboxes

Coalition for State Sandboxes
1100 G Street, NW, Suite 420, Washington, DC 20005