Dana Larew

Re: FDA-2026-N-7874

I am a registered nurse with hospital bedside experience. My questions concern the safeguards surrounding generative AI-enabled medical devices:

1. What tasks may these systems perform independently: reviewing records, interpreting results, developing care plans, entering orders, or initiating treatment? Where does their authority end?

2. Will qualified humans oversee development, testing, implementation, clinical use, and updates? Who are those people, what qualifications must they have, and who verifies their competence?

3. When AI creates or changes medication or treatment orders, who reviews and authorizes those orders before they reach the patient? Will that reviewer have enough time, clinical information, and authority to intervene?

4. When an intern or another clinician in training uses AI, who supervises the clinician and reviews the AI’s contribution?

5. If a patient has repeated low morning glucose readings and carbohydrate interventions, and AI recommends 32 units of insulin glargine at bedtime, who reviews the glucose trend, prior doses, nutrition, renal function, and nursing observations before authorizing the order?

6. How will the system recognize missing, outdated, contradictory, or wrong-patient information? What happens when it cannot support a safe recommendation?

7. For critical laboratory results, who must acknowledge the finding, authorize treatment, and confirm reassessment? What happens if that person is unavailable or fails to respond?

8. Can a bedside nurse pause a questioned AI-generated order and obtain prompt human review? Who resolves disagreement, and what protections exist for staff who raise safety concerns?

9. Are bedside nurses being consulted before purchasing and implementing these systems? Will testing include night shifts, interruptions, handoffs, realistic patient assignments, and competing emergencies?

10. What evidence shows that each tool is needed and improves patient outcomes compared with existing care? Are studies measuring preventable injuries, missed deterioration, treatment delays, unnecessary interventions, and nursing workload?

11. Who is represented in testing? How well do results apply to older adults, medically complex patients, and people underrepresented in the development data? Who independently verifies the findings?

12. When does testing or a hospital pilot constitute research involving patients? What ethics review, consent, and protections apply before patients are exposed?

13. Will patients be told when AI materially influences their diagnosis or treatment? Can they decline and receive a clinically appropriate alternative without an added charge or avoidable delay? What exceptions apply?

14. Who can access patient information? Can vendors retain it, sell it, or use it for further AI training? What choices do patients have?

15. Will implementation affect the hospital’s electrical capacity, Wi-Fi, network reliability, backup power, or other critical equipment? Who tests those effects before deployment?

16. What happens during a power failure, network outage, vendor outage, cyberattack, or delayed data feed? Who maintains safe care and prevents lost or duplicate orders?

17. If a nurse catches an unsafe AI recommendation before harm occurs, is that recorded as an AI error and a near miss? Who audits errors, overrides, and failures that never reach the patient?

18. Who monitors performance after deployment and after updates? What findings trigger suspension, recall, or renewed evaluation? Who has authority to stop use immediately?

19. If AI contributes to injury or death, who bears clinical, legal, and financial responsibility: the manufacturer, hospital, supervising prescriber, or bedside staff?

20. Who pays for the technology, training, oversight, infrastructure, maintenance, and failures? Will patients or taxpayers face added costs? How are injuries and additional bedside work included in claims of financial savings?

21. Could claimed efficiency gains lead to reduced staffing? What evidence would establish that enough qualified people remain available to provide meaningful oversight?

22. What approvals and readiness checks must precede implementation? What timeline is being considered, and which safeguards fall to FDA, hospitals, CMS, or professional licensing boards?

Dana Larew, BSN, RN

Human oversight and patient safety in generative AI medical
devices
Public comment on docket FDA-2026-N-7874
October 16, 2026

I am a registered nurse with hospital bedside experience. I ask FDA to make meaningful human
oversight, nursing participation, and demonstrable patient safety central to its approach to generative
AI-enabled medical devices. My central question is: Who is responsible for recognizing an unsafe AI
recommendation and stopping it before it reaches the patient?
The framework should distinguish assistance with information from authority to initiate or change
treatment. Oversight should extend across development, testing, deployment, clinical use, and updates,
with safeguards proportionate to the consequences of failure. Please address the following questions
and recommendations.

1 Identify who provides oversight and has authority to intervene
For a system that proposes or initiates high-risk medication or treatment orders, I request meaningful
review by an appropriately licensed prescriber before those orders are released, with pharmacist
verification and nursing safeguards appropriate to the intervention. Who supervises trainees using the
system? When is specialist review required? Identify the responsible clinical service on every shift and
clarify the responsibilities of the manufacturer, hospital, and supervising clinician when harm occurs.
If narrowly defined automated treatment functions are permitted without approval of each individual
action, require evidence supporting that specific use, explicit operating limits, continuous monitoring,
and a way for qualified staff to interrupt automation safely.

2 Require review of the patient and the clinical trend
Consider an illustrative patient whose morning glucose readings have repeatedly been in the 70s or low
80s mg/dL, with documented carbohydrate interventions. An AI system proposes 32 units of insulin
glargine at bedtime. That dose must be judged in the full clinical context. Will the reviewer see the
glucose trend, previous insulin administration, nutrition, renal function, and nursing observations, or
only the current diagnosis and latest result?
The 2026 ADA hospital standards identify fasting glucose below 100 mg/dL as a predictor of subsequent
hypoglycemia and discuss basal insulin adjustment. This supports evaluating trends before a patient
develops hypoglycemia; it does not determine the correct dose for an individual patient. [1]

3 Include bedside nurses before implementation
Bedside nurses are essential end users and must participate in design, procurement, testing,
implementation decisions, and ongoing safety review. Include night-shift nurses, newer nurses, and
nurses caring for medically complex patients. Test the system during interruptions, handoffs, competing
emergencies, and realistic assignments. Fund training and the time needed to review outputs. Any
staffing change justified by AI should require separate evidence that safe monitoring and intervention
remain possible.
4 Demonstrate benefit against an appropriate standard of care
What clinical problem does each device solve, and what evidence shows benefit compared with existing
care? Evaluate clinically important outcomes, including serious medication errors, missed deterioration,
delayed treatment, and preventable injury. Measure unnecessary interventions and added nursing work
as well as time saved. Publish the safety thresholds and the uncertainty around the results. A high
average accuracy score can conceal an unacceptable rate of rare, severe failures.

5 Establish who is represented in testing
Require evaluation on independent data and in representative care settings. Include older adults,
patients with multiple illnesses, renal or hepatic impairment, and other groups appropriate to the
intended use. Report results for these groups separately. For high-consequence treatment functions,
require prospective clinical evidence proportionate to risk and explain any decision to rely on less direct
evidence.
Use staged introduction with defined stop criteria. Clarify when a pilot is research, what independent
ethics review is required, and how applicable informed-consent protections will be met. Patients should
understand whether they are receiving established care or participating in an investigation.

6 Make independent clinical review possible
Show the patient data used, their source and time, relevant evidence, known limitations, and material
missing information. Test wrong-patient records, incorrect units, outdated results, and contradictory
documentation. Require the system to flag uncertainty and defer when safe action cannot be supported.
A plausible explanation is insufficient if the reviewer cannot verify the facts behind it. Another AI system
should not be the sole assessor of safety, especially when both systems may share failure modes.

7 Close the loop on critical results and nurse concerns
For critical results, specify who receives the finding, who acknowledges it, how quickly a qualified
clinician must respond, and what happens if no one responds. Follow the process through treatment
and reassessment. An alert being sent is not evidence that the patient received care.
Give nurses a clear process to pause a questioned AI-generated order and obtain timely human review,
consistent with scope of practice and emergency protocols. Protect good-faith safety escalation from
retaliation. Oversight that depends on an unavailable clinician or an overloaded nurse is inadequate.

8 Protect patient understanding and choice
Will patients receive plain-language notice when generative AI materially influences diagnosis or
treatment? Who explains its role, limitations, and the accountable clinician? Can a patient request a
clinically appropriate alternative without an added charge or avoidable delay? Specify how refusal is
documented, which emergency exceptions apply, and how patients with impaired decision-making
capacity are protected. Address consent for investigational use separately. General admission
paperwork should not be assumed to establish understanding of every subsequent AI use.

9 Validate infrastructure and protect patient information
Require hospitals to establish whether processing occurs on the device, on local servers, or in the cloud,
and test the actual electrical, network, Wi-Fi, and backup-power demands. What happens during a
network outage, vendor outage, cyberattack, or delayed data feed? Prevent lost or duplicate orders and
provide a practiced route back to safe clinical care. Test whether added traffic interferes with other
critical systems.
Specify who can access patient information, whether vendors retain it or use it for further training, and
how unauthorized access and manipulation of AI instructions will be detected and investigated.

10 Monitor failures and define when use must stop
Track harmful outputs, missed warnings, near misses, clinician overrides, and nurse interceptions after
deployment. If a nurse catches an unsafe recommendation before it reaches the patient, record the
underlying AI error rather than counting the encounter as proof of system safety. Preserve an auditable
record of the inputs, output, software version, human review, and action taken.
Require prompt investigation, notification, and predefined criteria for suspension or rollback. Reassess
safety after material model, workflow, or software changes. Clarify who can order an immediate local
stop and who must report problems to FDA.

11 Disclose who pays and the full cost of use
Who pays for acquisition, subscriptions, training, clinician oversight, cybersecurity, infrastructure,
maintenance, and failures? Will patients or public insurance programs face added charges? Disclose
financial relationships that could influence purchasing or evaluation. Include the costs of additional
tests, false alarms, treatment delays, and injuries in economic claims. Labor savings do not establish
clinical safety, and a financial benefit to an institution does not justify an undisclosed transfer of risk to
patients or bedside staff.

12 Define the requirements for deployment
What device-specific evidence and approvals must be in place before clinical use, and who confirms that
the hospital is ready? Publish the intended scope, permitted level of automation, training requirements,
limitations, and monitoring obligations. Identify which safeguards FDA can address and which require
coordination with CMS, professional licensing boards, or other authorities. Implementation timing
should follow demonstrated readiness and the requirements applicable to the particular device.
I ask FDA to translate human oversight into identifiable responsibility, adequate staffing, timely review,
and enforceable safeguards wherever its authority permits. Before an institution relies on AI to influence
treatment, patients and staff should be able to identify the responsible human, understand the evidence
supporting the system, and stop unsafe action. Bedside nursing judgment must be supported
throughout that process.
Dana Larew, BSN, RN

References
1 American Diabetes Association. Diabetes Care in the Hospital Standards of Care in Diabetes 2026. Diabetes Care
49 Supplement 1 S339 to S355.
FDA. Considerations for the Regulation of Generative AI Enabled Medical Devices.
FDA. Transparency for Machine Learning Enabled Medical Devices Guiding Principles.
FDA. Human Factors Considerations.