OCHIN (Jennifer Stoll)

OCHIN appreciates the opportunity to comment. Please find our comments attached.

Submitted via electronically Regulations.gov

October 19, 2026

Rick Abramson, MD
Director
Digital Health Center of Excellence (DHCoE)
Center for Devices and Radiological Health
U.S. Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993

Re: Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and
Request for Feedback (Docket No. FDA-2026-N-7874)

Dear Director Abramson:

OCHIN appreciates the opportunity to respond to the U.S. Food and Drug Administration (FDA) Center for
Devices and Radiological Health discussion paper on generative artificial intelligence (GenAI)-enabled medical
devices. As a national nonprofit health IT and shared services organization, OCHIN supports a consistent
national framework that enables us to evaluate, implement, and support safe and effective technologies across
our network. The framework must protect patients, support responsible innovation, and give our member
health care providers confidence to use authorized devices.

OCHIN provides shared technology, data, research, and implementation support to rural and low-resourced
community-based health care organizations nationwide. Our ability to support GenAI-enabled medical devices
depends on clear evidence, reliable information, and defined accountability across the product lifecycle. These
protections are especially important for our members, which often face workforce constraints, limited specialty
access, incomplete data, connectivity challenges, and fewer resources to evaluate and monitor emerging
technology on their own.

OCHIN: Shared Infrastructure for Rural and Community-Based Care
OCHIN supports more than 300 independent health care organizations, 44,000 providers across 44 states,
serving more than 10.5 million patients. Our members include federally qualified health centers, rural health
clinics, critical access hospitals, tribal health organizations, and other community-based providers. Through
shared health information technology, research, data, and implementation services, OCHIN sees how new
technologies interact with varied clinical workflows, staffing, data, connectivity, and technical capacity. That
network-level view also shows what our members need to adopt and use these tools safely in daily care.

Recommendations
FDA's framework should enable organizations to evaluate, implement, and support GenAI-enabled devices
across diverse care settings. This means it should work for the rural and community-based providers as well as
flagship medical centers. OCHIN recommends that the FDA:

• Include OCHIN and community-based/rural care settings from the start. FDA should encourage
manufacturers and study sponsors to involve shared technology and implementation partners such as
OCHIN in product design, clinical testing, and postmarket planning. They should also include the rural and
community-based providers that will use these devices in practice.
• Test devices through representative networks and care settings. FDA should recognize OCHIN and similar
provider networks as practical partners for clinical confirmation, local testing, and real-world evidence.
Benchmark results should support, not replace, testing with the patients, clinicians, workflows, and
resource conditions found across our member organizations.
• Keep manufacturers accountable. OCHIN can support implementation, reporting, and learning across the
network, but manufacturers must remain responsible for testing, monitoring, updates, investigations,
communication, and corrective action. OCHIN and its members should not be required to monitor or
correct technology they did not build and cannot inspect.
• Give OCHIN and its members information that matches the risk. As a device becomes more directive or
independent, FDA should require stronger evidence, closer monitoring, and greater human review. OCHIN
needs timely, actionable information about intended use, limits, data requirements, updates, performance,
and monitoring expectations so we can support safe implementation. Our members need clear guidance on
appropriate use, human review, escalation, and reporting.
• Create a clear path from authorization to safe use. FDA should build on its current risk-based device
pathways and provide a consistent national approach. FDA should also work with other federal agencies
and trusted partners such as OCHIN on implementation support, practical training, and reporting
infrastructure. Providers that use an authorized device as directed should have reasonable protections and
should not inherit responsibilities that belong to the manufacturer.

Additional details in the Appendix.

GenAI-enabled medical devices can help OCHIN and its members address workforce and access challenges, but
only if they work across the full range of settings in our network. Involving OCHIN early can strengthen testing,
implementation, and monitoring across diverse care environments. Including our rural and community-based
members ensures the framework reflects the realities of care delivery and gives providers confidence to adopt
authorized devices safely.

OCHIN welcomes continued engagement as FDA develops its regulatory framework. Please contact me at
stollj@ochin.org to discuss how we can support this work.

Sincerely,

Jennifer Stoll
Chief External Affairs Officer
 APPENDIX

IV. Considerations for the Assessment of Risk for GenAI-Enabled Devices

1. Risk Framework
Does the two-axis risk framework, organized around device activity and the consequence of relying on an incorrect output,
appropriately capture the dimensions most relevant to the risk of a GenAI-enabled software function? If there are additional
dimensions—such as the reversibility of a resulting action, the availability of downstream safeguards, the time pressure of the
deployment setting, or the traceability of the output (i.e., to primary source materials)—that should be represented in a risk framework,
please describe and provide examples of how they should be represented.
OCHIN supports the two-part framework as a useful starting point, but it does not capture all of the risks we see across our
network. FDA should also consider how quickly an output may affect care, whether an action can be reversed, whether a
clinician can review it first, and whether the output can be traced to the patient data and model version that produced it.
These factors matter differently in a clinic visit, an emergency department, an inpatient unit, and at discharge. The
framework should also account for missing data, weak connectivity, limited specialty backup, and whether the
organization has the staff and tools to identify and respond to an error.

2. Directiveness as a Continuum
CDRH seeks input on how to account for the spectrum of GenAI-enabled informational functions that vary in the degree to which they
direct a user to a particular action (i.e., between “non-directive” and “action-directing”). What characteristics of an output—such as its
wording, specificity, personalization, or context—could be considered as modifiers of the risk associated with an informational function,
after accounting for the device’s overall functionality and intended use? What additional information would provide manufacturers with
sufficient clarity and predictability around risk assessment for informational functions while recognizing that directiveness may exist
along a continuum rather than as a binary distinction?
FDA should treat directiveness as a sliding scale, not an on-off switch. An output becomes more directive when it is
specific, urgent, personalized, presented with high confidence, or placed directly into an order, care plan, patient message,
or discharge instruction. FDA should assess how a reasonable user is likely to understand and act on the output. Risk is
higher when a patient or clinician may act before another person can review it, especially in emergency, inpatient, afterhours, or rural settings where specialty support may not be readily available.

3. Patient-Facing Functions
CDRH seeks input on whether and when a GenAI-enabled function that results in delivery of clinical information to patients, as opposed
to HCPs, could present different or higher risks, while also recognizing the potential benefits associated with improved patient
empowerment, engagement, and access to clinical information. What device characteristics, output features, or safeguards might
mitigate risks that could arise when a user lacks the domain knowledge to independently evaluate an output, without unnecessarily
underestimating patient capability?
Patient-facing tools need safeguards that are clear and practical. They should explain what the output means, what
information was used, how certain the result is, when the patient should seek help, and how to reach a person. FDA
should also evaluate the work created for OCHIN members, including portal messages, follow-up calls, corrections, triage,
and care escalation. A tool that appears safe for one patient may create significant clinical and operational risk when used
by thousands of patients across ambulatory and hospital settings.

4. Generalist and Specialists Use
CDRH seeks input on whether and how the distinction between generalist and specialist physicians might inform the assessment of risk
for HCP-facing GenAI-enabled functions. Under what circumstances, if any, might risk be affected when an HCP who lacks the relevant
clinical specialist knowledge receives information that falls within a specialist area of practice? What device characteristics or safeguards
might mitigate such risks?
The key question is whether the intended user can recognize when the output is wrong, incomplete, or outside the user's
expertise. A tool intended for primary care, rural clinicians, emergency clinicians, hospitalists, nurses, or other generalists
should be tested with those users in their actual workflows. Specialist-only testing is not enough if the device will be used
where specialty review is delayed or unavailable. FDA should require clear limits, escalation guidance, and testing of
handoffs between ambulatory, emergency, inpatient, and specialty care.

5. Multi-Turn Conversations
For multi-turn conversational GenAI-enabled devices that may migrate from providing “non-directive” information to “action-directing”
information over the course of an exchange, how should risk be assessed across realistic conversational trajectories? How could the
intended use of such a device be characterized when its behavior is emergent across a conversation?
FDA should evaluate the full conversation, not just individual responses. A tool may begin by sharing general information
and later recommend a diagnosis, test, treatment, or urgent action. Manufacturers should test realistic conversations,
including changes in symptoms, incomplete answers, repeated questions, and conflicting information. The device should
clearly signal when the conversation becomes more directive, identify when human review is needed, and keep a record
that allows OCHIN, the care organization, the manufacturer, and FDA to reconstruct what happened.

6. Care Escalation
For GenAI-enabled care escalation functions, CDRH is considering how the evaluation may account for both under-escalation and overescalation. How could manufacturers characterize and weigh these two directions of error, given that they may not be commensurable
and that acceptable trade-offs may vary by clinical context?
Under-escalation and over-escalation should be measured separately. Missing a needed escalation can delay care or cause
harm. Too many escalations can overwhelm nurses, clinics, emergency departments, hospital teams, and scarce specialty
services. Manufacturers should test both types of error in the settings where the device will be used and report the effects
on patient safety, staff workload, wait times, transfers, follow-up capacity, and cost. Acceptable limits should be set before
testing and should reflect the clinical situation and available local resources.

V. A Competency-Based Approach for Premarket Evaluation of GenAI-Enabled Devices

7. Competency-Based Evaluation
Is the competency-based approach described above, i.e., device benchmarking followed by clinical confirmation, a useful and
appropriate framework for evaluating GenAI-enabled devices?
The evidence should match the risk. A device that summarizes information for clinician review may need less evidence
than one that recommends an order, changes a care plan, sends patient instructions, or takes action on its own. Greater
potential harm, less opportunity for human review, or harder-to-reverse actions should require stronger independent
testing, real-world clinical confirmation, and closer monitoring. FDA should allow different evidence approaches for
different uses, but it should not lower the basic safety standard because a device will be used in a lower-resourced setting.

8. Risk-Proportionate Evidence
How might the two-axis risk framework described in Section IV be considered within the competency-based approach described above
to help determine the level of evidence needed for a premarket submission?
The proposed areas are reasonable, but benchmarks must match the device's intended job. They should test more than
medical knowledge. They should also test how the complete device handles missing or conflicting data, communicates
uncertainty, works with clinicians and patients, and performs across ambulatory visits, emergency care, inpatient care,
discharge, and follow-up. Sponsors should disclose the data used, known failure patterns, and important gaps. Good
model performance does not by itself show that the full product will work safely in an OCHIN member's workflow.

9. Benchmarking Structure
Would a benchmarking structure such as the one described above be likely to provide adequate evidence of clinical knowledge, analytic
capabilities, safety behavior, communication, and generalizability to support a reasonable assurance of safety and effectiveness? Are
there elements that are missing, 17 redundant, or inappropriately categorized? Are there externally developed standards that could be
leveraged?
A sponsor should show that a benchmark reflects the patients, tasks, and care settings in which the device will be used.
This includes the kinds of incomplete records, changing conditions, data from outside organizations, and workflow
pressures common across OCHIN's network. If rural, safety-net, tribal, hospital, or ambulatory settings are not well
represented, the sponsor should disclose the gap and address it through additional testing, labeling, or monitoring.
Sponsor-developed benchmarks should be independently reviewed when they are used to support important safety
claims.

The testing approach should match how much harm an error could cause, how independently the device acts, whether the
action can be reversed, and whether a clinician can review the output before it affects care. Higher-risk, patient-facing,
action-directing, or autonomous devices should generally be tested prospectively in representative settings. Lower-risk
tools may be supported by retrospective review or silent testing when reliable human review remains in place. FDA should
allow testing through networks such as OCHIN so evidence can include ambulatory clinics, rural hospitals, emergency
departments, inpatient units, and transitions between settings.

10. Benchmark Validity
CDRH recognizes that publicly available benchmarking assets may be subject to data contamination, saturation, and limited real-world
representativeness. How should a sponsor establish that performance on a given benchmark predicts safe and effective real-world
behavior for the device’s intended use? What evidence should support the construct validity of a benchmark used to gate device
evaluation, and what role should sponsor-developed benchmarks play given potential concerns around independence and optimization
to the test?
Sponsors should decide in advance what success means, how much uncertainty is acceptable, and how many cases are
needed. Results from real patients and synthetic data should usually be reported separately so synthetic cases do not hide
gaps in real-world evidence. Performance should be reported by care setting, patient group, and intended user when
differences could affect safety. FDA should also look beyond average accuracy to clinically important errors, workload,
false alerts, missed escalations, and performance when data are incomplete.

11. Clinical Confirmation
CDRH is considering that clinical confirmation for a GenAI-enabled device might not require a prospective clinical study in every case,
and has described above a range of approaches of increasing rigor and patient exposure. How might a sponsor select and justify a
confirmation approach tailored to a device’s intended use and proportionate to the device’s risk profile? Are there device types or risk
profiles for which one or more of these approaches would be insufficient or inappropriate? How might the anticipated distribution of
real-world inputs be taken into consideration? Are there other methods of clinical confirmation that might help inform the evaluation of
GenAI-enabled devices?
Synthetic data can help test rare events, unusual combinations of conditions, system failures, or situations that are difficult
to study at scale. It should not replace real-world evidence from the patients and settings OCHIN serves, including rural,
uninsured, publicly insured, tribal, and other medically underserved populations. Sponsors should compare synthetic cases
with real data, explain where they differ, and test whether the same type of model used to create the synthetic data may
repeat the blind spots of the device being evaluated.

12. Performance Measurement and Synthetic Data
How should sponsors achieve statistically meaningful performance measurement for GenAI-enabled devices? Where synthetically
generated inputs supplement real patient data, how should sponsors account for differences between the synthetic and real-world
distributions when estimating performance, and under what conditions, if any, is it appropriate to combine benchmarking evidence and
clinical confirmation evidence to support a single performance estimate?
The comparator should match the real decision and workflow. In some cases that may be a specialist. In others it may be a
primary care clinician, emergency clinician, hospitalist, nurse, care team, delayed specialist review, or the care likely
without the device. When a device is designed to support a person, FDA should evaluate the person and device together.
Success should include safety, usefulness, clear communication, time saved or added, false alerts, missed findings, and
performance across relevant patients and care settings.

13. Appropriate Uses of Synthetic Data
For which clinical domains, device functions, or subpopulations is synthetic data particularly well-suited, or particularly inadequate, as a
supplement to real world evidence? What safeguards would mitigate the risk that synthetic data generated by models of the same class
as the device under evaluation reproduces the very performance gaps the evaluation is intended to detect, particularly for
underrepresented subgroups?
FDA should allow comparison with the care that would realistically occur without the device. Across OCHIN's network, that
might be primary care management, an emergency department transfer, delayed specialist input, or no timely
intervention. Sponsors should explain why the comparator fits the intended setting and measure effects on access,
timeliness, workload, transfers, follow-up, and outcomes. The absence of a local specialist or other resource should not
lower the minimum safety standard.

14. Comparators and Acceptance Criteria
For open-ended device outputs, how should performance comparators and acceptance criteria be selected? When a panel of qualified
clinicians serves as the comparator, how should the applicable standard (for example, the standard of care versus the performance of a
median clinician in practice) be defined and justified? Should generalist or specialist physicians be used as a performance standard?
When should human-AI team performance, rather than the device operating alone, serve as the basis for evaluation?
Independent third parties can strengthen testing and audits for higher-risk devices. They should have clinical, technical,
data, implementation, and health-system expertise, including experience in both ambulatory and acute care. They should
disclose conflicts and use transparent methods. FDA should recognize several qualified pathways, including nonprofit
provider networks, multi-site research networks, and neutral research partners, so independent review does not become
too costly or limited to a small group of evaluators.

15. Real-World Alternatives
Are there ways in which performance might be assessed relative to the care, technology, or course of action likely to occur in the
absence of the device, rather than to the comparators described in this section? What approaches might be used to identify and justify a
comparator such as unaided clinical judgment, delayed specialist review, or no intervention?
Devices that combine text, images, audio, video, monitoring data, or outside tools should be tested as complete systems.
FDA should examine what happens when inputs conflict, an image is poor, data are missing, a connected service is
unavailable, or the patient's condition changes over time. Testing should cover the full workflow, including handoffs
between clinic, emergency, inpatient, discharge, and follow-up care. The device should fail safely and clearly alert the user
when it cannot produce a reliable result.

16. Independent Third Parties
Should independent third parties be involved in some or all aspects of a competency-based approach, including device benchmarking
and clinical confirmation? If so, in what ways might qualified, independent third-party 18 participation contribute to this approach, and
what qualifications and independence criteria should apply? Are there aspects of the assessment for which third-party involvement
would be impractical or inadvisable? What safeguards or program design features would be critical to prevent third-party participation
from limiting competition or preventing innovation?
Independent third parties can strengthen testing and audits for higher-risk devices. They should have relevant clinical,
technical, and health-system expertise, disclose conflicts, use transparent methods, and remain independent from the
sponsor. FDA should allow several qualified pathways, including federated provider networks and neutral research
partners, rather than create a small and costly evaluator market.

17. Multimodal and World-Model Devices
CDRH is exploring whether a competency-based approach could be applied to devices that incorporate a variety of underlying model
architectures, such as multimodal vision-language models and generative or predictive world models. Are there aspects of the described
approach that might be ineffective or inapplicable to these devices? What additional or distinct considerations might need to be
addressed?
FDA should accept less evidence before authorization only for lower-risk uses when errors are reversible and dependable
human review remains in place. This approach is not appropriate for high-risk, action-directing, autonomous, or hard-toreverse uses. Any greater reliance on postmarket monitoring should come with a detailed manufacturer plan, timely
access to performance information, clear triggers for action, and the ability to stop or roll back the device. Manufacturers
should fund and support the monitoring they require rather than shifting the cost and workload to OCHIN or our
members.

VI. Postmarket Monitoring for GenAI-Enabled Devices

18. Premarket Uncertainty and Postmarket Monitoring
CDRH is considering whether it may be appropriate to accept greater premarket uncertainty regarding a GenAI-enabled device’s benefitrisk profile through greater reliance on postmarket monitoring. Under what conditions might such an approach be appropriate, and
what characteristics of a monitoring program would need to be in place to justify reduced premarket evidence? Are there device types
or risk profiles for which this approach would not be appropriate?
FDA should require periodic retesting, review of selected cases, monitoring for performance decline, adverse-event
reporting, and evaluation across patient groups and care settings. The schedule should reflect the risk and should
accelerate after changes to the model, training data, EHR workflow, clinical scope, outside tools, security controls, or use
environment. Plans should state when the manufacturer must investigate, notify OCHIN and affected members, correct
the device, roll back a change, or suspend use. Health systems also need useful feedback when they report a concern.

19. Postmarket Evaluation
Please comment on the potential approaches to postmarket performance evaluation, including periodic re-benchmarking, sample-based
clinician review, and performance degradation monitoring. What additional approaches should CDRH consider, and how should the
cadence and triggering events for reassessment be determined?
One AI system may help monitor another, but it should not be the only safeguard. The monitoring system must be tested
separately, including whether it shares the same blind spots or dependencies as the device it monitors. It should have
version control, audit logs, clear thresholds, and a defined path to human review. OCHIN and our members also need to
know who receives an alert, how quickly it must be reviewed, and what happens if the monitoring system or a connected
service fails.

20. Machine-Based Supervisory Agents
Please comment on whether the proposed approaches to postmarket monitoring can be facilitated by machine-based supervisory
agents. What considerations, including the evaluation and reliability of the supervisory agent itself, should CDRH take into account for
such an approach?
An AI system may help monitor another AI system, but it should not replace human and organizational oversight. The
monitoring system should be tested independently and checked for its own failures, including risks shared with the device
it monitors. It should also have version controls, audit logs, clear escalation rules, and human review.

21. Stakeholder Roles and Manufacturer Accountability
What roles might clinicians, healthcare institutions, professional societies, standards-setting bodies, and other stakeholders
appropriately play in postmarket monitoring, and how can these roles be structured without diffusing manufacturer accountability?
OCHIN and similar network organizations can help aggregate reports, assess implementation and workflow effects, and
contribute real-world data across care settings. Our member clinicians and health care organizations can identify failures
and describe their effects on care delivery. Professional groups can help define measures and reporting standards.
Participants need simple reporting channels, practical support, and feedback on reported concerns. Manufacturers should
remain responsible for surveillance, investigation, communication, corrective action, and the resources needed to monitor
their devices.

22. Reassessment After Modifications
CDRH envisions that a manufacturer’s premarket competency-based assessment might serve as a baseline against which postdeployment modifications could be re-evaluated. How might the extent of re-benchmarking or other evidence be scaled to the nature
and expected impact of a given modification? For example, are there categories of change that might not significantly affect safety or
effectiveness of a GenAI-enabled device, or might be appropriately managed within a sponsor’s quality management system versus
requiring FDA premarket review and authorization before 21 implementation? Are there categories of changes that might be
appropriate for inclusion in a PCCP?
The amount of retesting should match the likely impact of the change. Minor administrative changes may be handled
through the manufacturer's quality system. Changes to the model, training data, clinical scope, level of autonomy, outside
tools, EHR integration, safety controls, or patient-facing communication may change how the device performs and should
require more testing. FDA should also consider whether a change affects ambulatory and acute care differently or creates
new risks during handoffs and transitions.

23. Predetermined Change Control Plans
How might PCCP concepts or other change-control approaches be adapted for GenAI-enabled devices when the nature or scope of
future modifications cannot be fully prespecified?
A predetermined change control plan should set clear boundaries. It should describe which changes are allowed, how
large they may be, how they will be tested, what results must be met, who must be notified, how deployment will be
controlled, and when the change will be stopped or rolled back. OCHIN and our members need advance notice of material
changes and enough information to assess workflow, training, technical, and patient-safety effects. Changes outside the
approved boundaries should return to FDA for review.

24. Third-Party Foundation Model Changes
For devices built on third-party foundation models, changes to the underlying model may be initiated by the third-party model
developer rather than the device manufacturer. How can a manufacturer detect, evaluate, and respond to such changes in a timely
manner? What mechanisms—for example, contractual, technical, or through a PCCP—could provide reasonable assurance that thirdparty developer-initiated changes do not compromise the safety or effectiveness of the device?
Manufacturers should prevent an untested foundation-model update from reaching clinical use. They should know which
model and version is running, detect outside changes, test the complete device after a change, and be able to hold, roll
back, or suspend deployment. Contracts with model suppliers should require advance notice and access to safety-relevant
information. The medical device manufacturer must remain responsible for the complete product, including its
performance in OCHIN's ambulatory and acute care environments.

VII. Other Topics

25. Foundation Model Device Master Files
Would voluntary Foundation Model MAFs be practical for foundation model developers to provide and to keep current, and what
content might they include? Given that participation would be voluntary and that model developers may have limited incentive to
disclose safety-relevant information, what would make such a program sufficiently useful for premarket review? Are there alternative
mechanisms CDRH should consider for obtaining information about underlying foundation models?
Foundation Model Device Master Files could help FDA review devices more consistently if they are current and contain
useful safety information. They should identify the model and version, supported and excluded uses, health care testing,
known failure patterns, safeguards, update practices, and notice of material changes. FDA should consider requiring this
information for autonomous and other higher-risk systems when it cannot otherwise assess safety. A master file should
not replace testing of the actual device or reduce the device manufacturer's responsibility.

26. Agentic AI
Are there additional considerations that inform the premarket and postmarket evaluation of agentic GenAI-enabled devices, beyond
those applicable to non agentic GenAI-enabled devices? How should the elevated risk associated with autonomous multi-step action,
tool use, and reduced opportunity for human review be reflected in acceptance criteria and oversight?
Agentic devices need closer review because one error can spread through several steps, reach outside tools, or affect care
before a person can intervene. Testing should cover planning, tool selection, access permissions, orders and messages,
handoffs, error recovery, misleading instructions, and required human approvals. High-risk or hard-to-reverse actions
should require human approval unless FDA has strong evidence that independent action is safe. The device must stay
within its authorized role, stop safely when data or services fail, and keep a complete record so OCHIN, our members, the
manufacturer, and FDA can reconstruct what happened.