Michelle Bernabe, RN, BSN

i am a Columbia University-trained registered nurse with more than a decade of frontline experience, including psychiatric emergency care. i write Moral Health about narrative medicine, health reform, and responsible innovation. i submit in my individual capacity, not for National Nurses United. my recommendations align with NNU’s Nurses and Patients’ Bill of Rights.

i am an innovation enthusiast. generative ai may help patients understand and participate more fully in their care. it may help clinicians synthesize complex information, uncover patterns that might otherwise be missed, support earlier and more personalized interventions, strengthen communication and care coordination, reduce administrative burdens, extend expertise to underserved communities, and accelerate research and discovery. some of its most meaningful uses may still be waiting to be imagined. but innovation in health care must begin with safety. the people affected by these systems are our friends, partners, parents, children, neighbors, and eventually ourselves. when an ai-generated output enters care, it enters a relationship of trust among a patient, family, nurse, and clinical team.

1. build the risk framework around real conditions of care

in response to questions 1 through 6, FDA should assess both function and real conditions of use: reversibility, urgency, vulnerability, error detection, traceability, scale, staffing, and workflow. risk includes missed and unnecessary warnings. classification should reflect what happens when a system is wrong and whether clinicians have the time, information, authority, and staffing to respond safely.

2. require real clinical evidence for high-risk systems

in response to questions 10 through 16, benchmarks alone are not enough for systems that influence diagnosis, triage, treatment, escalation, or monitoring. high-risk systems need prospective clinical validation across diverse patients and settings.

FDA should evaluate the human-AI team, including registered nurses, not only the model. testing should measure hallucinations, omissions, automation bias, verification time, cognitive burden, workflow disruption, overrides, delays, and near misses. comparisons should be against safe, adequately staffed care, not an understaffed environment that makes a product appear beneficial.

3. protect clinical relationships and professional judgment

ai should support relationships, not displace them. patients need a real person who can listen, explain, notice what does not fit, and accept responsibility. nurses and other clinicians must be able to question, override, document concerns about, and stop unsafe ai recommendations without retaliation.

an ai tool should never substitute for safe staffing. patients and clinicians should know when ai shapes care and who remains accountable. disclosure matters, but disclosure alone is not meaningful consent and does not make an unsafe system safe.

4. use postmarket monitoring to supplement strong premarket evidence

in response to questions 18 through 21, postmarket monitoring is essential because performance can change across populations, institutions, workflows, and time. it must supplement strong premarket evidence, not replace it. patients should not become involuntary test subjects.

monitoring should cover subgroup performance, adverse events, near misses, changing errors, and protected frontline reporting. a machine should not be the sole supervisor of another machine. manufacturers must remain responsible for surveillance, correction, and transparent communication.

5. place enforceable controls on model changes

in response to questions 24 and 26, every clinically relevant output should be traceable to a specific model version. material changes should be detected before they affect care, followed by appropriate revalidation and fda review. health systems need prompt suspension and rollback controls. agentic systems that plan, initiate actions, or interact with other systems require a higher standard because errors can propagate before a person can intervene.

i want us to pursue innovation with imagination, curiosity, and purpose. it can help make care safer, more humane, and more accessible. that pursuit must also account for competitive pressure, cost savings, and incentives to scale quickly, especially when the people who benefit are not the same people who bear the risks. a balanced framework should create proportionate pathways requiring strong evidence, transparency, human accountability, and enforceable safeguards when the stakes are high. safety is not opposition to innovation. safety is how we make innovation worthy of personal and collective trust.

when the patient is our friend, our partner, our parent, our child, or ourselves, human alignment must come first. those who benefit from ai must not transfer its risks to patients, families, nurses, and clinical teams.

michelle bernabe, RN, BSN
writer, Moral Health