Mark Smith

I am submitting the attached comments in response to Docket No. FDA-2026-N-7874, “Considerations for the Regulation of Generative AI-Enabled Medical Devices."

I am a registered nurse with a background in both clinical practice and business. My comments focus on an issue that I believe warrants greater attention in the proposed framework: generative and agentic AI-enabled medical devices should be evaluated not only for the accuracy of their outputs, but also for their effects on the clinical workflows in which those outputs are interpreted, validated, corrected, escalated, documented, and acted upon.

Five recommendations are developed in the attached comment:

FDA should determine the appropriate clinical comparator according to the clinical function being performed, rather than defaulting primarily to generalist or specialist physicians. In many use cases, the appropriate comparator may be a registered nurse, advanced practice nurse, pharmacist, respiratory therapist, or interdisciplinary clinical team.

Evaluation should extend beyond model accuracy to include the performance of the AI-enabled clinical workflow. Measures should include clinician overrides and corrections, false or unnecessary escalations, missed escalations, verification burden, alert burden, workflow steps added or removed, and downstream patient outcomes.

Nurses and other frontline clinicians should be explicitly incorporated into postmarket surveillance because they are often positioned to identify erroneous recommendations, inappropriate alerts, incorrect generated documentation, near misses, and workflow failures. Manufacturers should remain responsible for collecting and analyzing these signals rather than transferring quality-control responsibilities to frontline staff.

For agentic AI, “human oversight” should be operationally defined. For high-consequence actions, the responsible clinician should have sufficient information, time, authority, and system transparency to meaningfully interrupt or override an AI action.

FDA should consider whether claimed efficiency benefits represent true reductions in clinical work or merely redistribute work to other clinicians. AI-generated work that subsequently requires verification, correction, escalation, documentation, or patient explanation remains part of the technology’s real clinical and economic footprint.

The central principle of my comment is that the appropriate unit of evaluation should not always be the AI model in isolation. For many medical AI applications, it should be the AI-enabled clinical workflow.

A system can produce technically accurate outputs while simultaneously increasing cognitive burden, creating new failure modes, shifting work to nurses or other clinicians, or requiring substantial human intervention to remain safe. Those consequences are material to both device effectiveness and patient safety.

The attached document provides the complete discussion and recommendations.

Respectfully submitted,

Mark Smith, MBA, BSN, RN
Submitted in an individual capacity

Public Comment on Considerations for the Regulation of Generative AI-Enabled Medical
Devices

Docket No. FDA-2026-N-7874

Submitted by:
Mark Smith, MBA, BSN, RN
Registered Nurse
Submitted in an individual capacity
Introduction

I appreciate the opportunity to comment on the Food and Drug Administration's discussion
paper, Considerations for the Regulation of Generative AI-Enabled Medical Devices.

I am a registered nurse with a background in clinical practice, business, and healthcare
operations. I submit these comments in my individual capacity. They do not represent the views
of my employer or any organization with which I am affiliated.

Generative artificial intelligence and agentic AI have substantial potential to improve healthcare.
They may reduce administrative burden, improve access to information, support clinical
decision-making, identify deterioration earlier, streamline communication, and enable new forms
of patient support.

At the same time, the safety and effectiveness of these technologies cannot always be adequately
evaluated by measuring the performance of the artificial intelligence model in isolation.

In many healthcare settings, the clinically meaningful unit of evaluation will be the AI-enabled
clinical workflow: the combination of the device, the clinicians interacting with it, the
information available to those clinicians, the actions that follow its outputs, and the
organizational environment in which those actions occur.

This distinction is especially important for nursing.
Nurses frequently occupy the point in the healthcare system where information becomes action.
They monitor patients, interpret changes in condition, validate information from multiple
sources, administer treatments, educate patients and families, coordinate care, identify
inconsistencies, escalate concerns, and intervene when expected processes fail.

As generative and agentic AI systems enter these workflows, nurses and other frontline clinicians
may increasingly become part of the safety architecture surrounding those technologies.

FDA's regulatory framework should recognize and measure that role.

My comments focus primarily on:

• Question 14 regarding clinical comparators and human-AI team performance;
 • Questions 19 and 21 regarding postmarket monitoring and stakeholder roles;

• Question 26 regarding agentic AI; and

• the broader need to incorporate clinical workflow consequences into assessments of
safety, effectiveness, and benefit.

I. The Appropriate Clinical Comparator Should Be Determined by the Clinical Function
Being Performed

Response to Question 14

Question 14 asks how performance comparators should be selected for open-ended outputs,
including whether generalist or specialist physicians should serve as performance standards and
when human-AI team performance should form the basis of evaluation.
I encourage FDA to broaden this framework.

The appropriate clinical comparator should be determined by the clinical function being
performed and the intended user of the device, rather than by assuming that physician
performance is the appropriate clinical reference point.

Modern healthcare is delivered by interdisciplinary teams. Many clinically consequential
functions are performed primarily or substantially by registered nurses, advanced practice
registered nurses, pharmacists, respiratory therapists, therapists, technicians, and other healthcare
professionals.

Consider potential AI-enabled functions involving:
• patient triage;

• monitoring for clinical deterioration;

• medication administration support;

• discharge education;

• patient messaging;

• care coordination;

• nursing documentation;
• clinical handoffs;

• fall-risk assessment;

• pressure-injury prevention;

• infection surveillance;
 • symptom assessment;

• escalation recommendations;

• prioritization of clinical tasks; and

• interpretation of changes in patient status.
A physician may not be the most appropriate comparator for many of these functions.

If an AI-enabled device is intended to perform, augment, influence, or partially automate work
ordinarily conducted by a registered nurse, then appropriately qualified nurses should be
considered among the relevant comparators.

Likewise, where an AI-enabled system is designed to operate as part of an interdisciplinary
process, evaluation against the performance of the clinical team with and without the
technology may be more informative than evaluation of the device against an individual
physician.

FDA should therefore consider establishing the following principle:

The comparator for an AI-enabled medical device should reflect the profession, clinical
function, environment, and workflow against which the device is actually intended to
operate.

This approach would more accurately reflect real-world clinical practice and reduce the risk of
demonstrating performance against a comparator that does not represent the device's actual use
environment.
II. Human-AI Team Performance May Be More Important Than Standalone Model
Performance
The distinction between model performance and clinical-system performance is particularly
important for generative AI.

A system can produce an output that appears accurate in isolation while creating new problems
when embedded in a real clinical environment.

For example, an AI system might correctly identify potential deterioration but produce so many
low-value alerts that clinicians begin disregarding them.

A documentation system might generate clinically plausible notes but require substantial
clinician time to verify, edit, or reconcile them with the medical record.

A patient communication system might provide generally accurate information while creating
confusion that subsequently requires clarification by nurses.
A clinical decision-support tool might perform well in controlled testing while causing delays
because clinicians must repeatedly investigate recommendations that do not fit the patient's
actual condition.

In each example, conventional measures of accuracy may suggest that the technology performs
well.

The clinical workflow may tell a different story.

FDA should therefore encourage evaluation at three levels when appropriate:

1. Device-level performance: Is the AI output accurate, reliable, and appropriately
calibrated?

2. Human-AI interaction performance: Can the intended user understand, validate,
appropriately rely upon, challenge, and override the output?

3. Clinical-system performance: Does use of the system improve the overall process and
outcome of care compared with the likely alternative?

This distinction becomes increasingly important as AI moves from passive information
generation toward recommendation, prioritization, planning, and autonomous action.

III. Clinical Accuracy Alone Is Not Sufficient Evidence of Real-World Safety or
Effectiveness

A GenAI-enabled device may be technically accurate while still imposing significant burdens or
creating new clinical risks.
FDA should consider whether postmarket and, where appropriate, premarket evaluations should
include measures beyond conventional accuracy.
Potential measures include:
• clinician correction rates;

• clinician override rates;

• frequency of false-positive recommendations;

• frequency of false-negative recommendations;

• unnecessary escalation;

• missed or delayed escalation;

• alert burden;
• interruption burden;
 • verification time;

• correction time;

• documentation added to the workflow;

• documentation eliminated from the workflow;
• duplicate work generated by the device;

• changes in time required to complete clinical tasks;

• frequency of clinician workarounds;

• frequency of recommendations abandoned after clinician review;

• near misses;

• downstream interventions triggered by AI outputs;

• patient outcomes associated with those interventions; and

• circumstances in which human review prevents an AI-generated error from reaching the
patient.

These measures are not merely questions of user satisfaction.
They may be directly related to safety.

Clinical attention is a finite resource. A system that unnecessarily consumes clinician attention
can potentially reduce the attention available for other patients and tasks.

Similarly, a system that requires frequent correction may create a safety dependency that is
invisible if evaluation focuses only on whether the final clinical outcome was correct.
IV. The Hidden Human Safety Layer Should Be Measured

One of the most important issues FDA should consider is the possibility that AI-enabled systems
will depend upon substantial amounts of human work that are not currently captured as part of
device performance.

When a generative AI system produces an incorrect or questionable output, the healthcare system
may remain safe because a clinician notices the problem.

A nurse may recognize that an output conflicts with the patient's current condition.
A pharmacist may recognize an inappropriate medication recommendation.

A physician may identify an incorrect diagnostic interpretation.
A respiratory therapist may recognize that a recommended intervention is inconsistent with the
patient's physiology.

When that occurs, the eventual absence of patient harm does not necessarily demonstrate that the
AI system performed safely.

It may demonstrate that the human safety layer surrounding the system performed
successfully.

Those are different conclusions.

For generative AI systems, FDA should encourage manufacturers and healthcare organizations to
distinguish between:

• errors the device prevented;
• errors the device introduced;

• errors detected automatically;

• errors detected by clinicians before reaching patients;

• errors that reached patients but did not result in harm; and
• errors that resulted in clinical consequences.

A near miss detected by a nurse should not disappear from performance data simply because the
nurse successfully protected the patient.

Those events may contain some of the most valuable information available for identifying
recurrent weaknesses in an AI-enabled medical device.
V. Nurses Should Be Explicitly Incorporated Into Postmarket Surveillance

Response to Questions 19 and 21

Postmarket surveillance will be especially important for generative AI because performance may
vary with context, user behavior, patient population, workflow, system updates, and the
distribution of real-world inputs.
Nurses should be explicitly considered in the design of these monitoring systems.

Nurses frequently interact with patients continuously across shifts and across multiple
components of care. As a result, they may observe failures that are not easily detectable through
automated performance monitoring.

Examples may include:
• an AI recommendation that appears reasonable but is inappropriate for the patient's
current condition;
 • generated documentation that mischaracterizes what occurred;

• an alert that repeatedly activates under clinically benign circumstances;

• a system that fails to recognize meaningful changes that nurses identify through direct
assessment;

• recommendations that conflict with current orders or treatment plans;

• patient-facing information requiring repeated clarification;

• workflow changes that create unsafe delays;
• outputs that increase rather than decrease cognitive burden; and

• recurring workarounds developed by frontline staff because the technology does not fit
actual care processes.

FDA should encourage postmarket surveillance systems that make these signals easy to capture
and aggregate.
Such systems should include mechanisms for reporting:

• adverse events;

• near misses;

• overrides;

• corrections;

• inappropriate recommendations;

• clinically significant omissions;
• recurrent workarounds;

• unexpected workflow consequences; and

• patterns of disagreement between AI outputs and frontline clinical judgment.

However, FDA should also guard against a significant unintended consequence.

The responsibility for postmarket safety surveillance should not simply be transferred to frontline
clinicians.

If an AI-enabled device requires substantial ongoing clinician review to identify hallucinations,
inappropriate recommendations, errors, or performance degradation, that review represents part
of the operational cost of the technology.
Manufacturers should remain responsible for developing mechanisms to capture, analyze,
respond to, and appropriately report safety signals associated with their products.

Clinicians should contribute to the safety system.

They should not become an invisible or uncompensated quality-control system for commercial
technology.

VI. FDA Should Distinguish True Automation From Work Redistribution

Healthcare organizations and technology manufacturers frequently describe AI in terms of
productivity, efficiency, automation, and reduction of clinician burden.

Those claims can be meaningful, but they can also be misleading if measurement stops at the
automated task.
Consider a hypothetical system that reduces ten minutes of documentation work for one clinician
but creates several minutes of verification, correction, escalation, communication, or
reconciliation work elsewhere in the care process.

The system may still provide a net benefit.

But the size and location of that benefit are different from what would be suggested by
measuring only the original automated task.

This leads to an important regulatory and health-system question:

Does the AI system eliminate clinical work, or does it redistribute that work?

That distinction is especially relevant to nursing because nursing work has historically been
difficult to isolate within healthcare accounting structures.

Much of nursing's economic contribution is embedded within hospital operations rather than
separately billed as discrete professional services. As AI systems increasingly enter clinical
workflows, there is therefore a risk that additional nursing work required to supervise, validate,
correct, or operationalize AI outputs could remain economically invisible.
FDA does not need to become a regulator of healthcare labor economics to recognize this issue.

Rather, when efficiency, burden reduction, productivity, or workflow improvement is relevant to
the claimed benefit or risk profile of an AI-enabled medical device, those outcomes should be
measured across the affected workflow.

Metrics might include:
• total clinician time before and after implementation;

• distribution of work among professional groups;
 • time spent validating AI outputs;

• time spent correcting AI outputs;

• time spent investigating false alarms;

• additional communication generated by the device;
• changes in task switching and interruptions;

• work transferred from one professional group to another; and

• whether time savings observed in one portion of the workflow are offset elsewhere.

A technology should not be characterized as reducing clinical burden when the burden has
primarily been moved to a less visible part of the healthcare workforce.
VII. Agentic AI Requires a More Operational Definition of Human Oversight

Response to Question 26

Agentic AI systems present a qualitatively different challenge because they may not merely
generate information. They may plan, sequence, initiate, or execute actions.

As autonomy increases, the concept of "human oversight" should be defined with greater
precision.

The existence of a human somewhere within the workflow should not by itself constitute
meaningful oversight.

For high-consequence actions, meaningful human oversight should generally require that an
appropriately qualified clinician has:

1. sufficient information to understand what the system proposes to do;

2. sufficient context to evaluate why the proposed action is occurring;

3. sufficient time to intervene before an irreversible or high-risk action takes place;

4. explicit authority to stop, modify, or override the action;

5. access to a record of the information used by the AI system;
6. visibility into relevant actions the system has already taken;

7. a clear mechanism for identifying uncertainty or limitations in the system's
recommendation;

8. an accessible escalation pathway when the system behaves unexpectedly; and

9. documentation sufficient for subsequent clinical and safety review.
FDA should also consider differentiating agentic actions according to reversibility and potential
consequence.

A system drafting a patient education message presents a different risk profile from a system
autonomously modifying therapy, prioritizing patients, suppressing alerts, initiating orders, or
communicating consequential clinical recommendations.

As the potential severity and irreversibility of an action increase, the threshold for meaningful
human authorization should increase as well.
In many clinical settings, the appropriate human checkpoint may be a registered nurse rather than
a physician.
The regulatory framework should allow the oversight role to follow the clinical workflow while
ensuring that the person responsible has the training, authority, information, and opportunity
necessary to intervene effectively.

VIII. Human Oversight Itself Should Be Evaluated as a Safety-Critical Function

The phrase "human in the loop" can create a false sense of security unless the human component
is evaluated.

A safety design that assumes clinicians will detect AI failures should test whether clinicians can
actually do so under real working conditions.

Relevant questions include:

• Is the AI's output distinguishable from verified clinical information?
• Does the user understand the system's degree of uncertainty?

• Can the user identify the information on which the recommendation was based?

• Can the clinician challenge or override the recommendation easily?

• Does the system create automation bias?

• Does repeated exposure to largely correct outputs reduce vigilance?

• Is there sufficient time to review outputs during realistic workloads?

• Does the interface make disagreement more difficult than agreement?
• Are users penalized operationally for overriding the system?

• Can clinicians recognize when the system is operating outside its validated context?

• Do workload, staffing, fatigue, and interruptions materially affect the effectiveness of the
human safety layer?
If the safety case for a device depends on clinician supervision, then the effectiveness of that
supervision is part of the device's real-world safety architecture.

It should be evaluated accordingly.

IX. Postmarket Monitoring Should Capture Drift in Workflow, Not Only Drift in Model
Performance

Performance drift is commonly discussed as a change in the relationship between model inputs
and outputs over time.

For clinical AI, there may also be workflow drift.

Healthcare organizations adapt around technologies.

Clinicians create shortcuts.

New policies are introduced.

Responsibilities shift between disciplines.
Training quality changes.

Staff turnover occurs.

Alert thresholds are modified.

Additional technologies are added to the same workflow.

Clinicians may begin relying on an AI system differently as trust increases or decreases.

A system that was initially safe under close observation may therefore interact differently with
the clinical environment months or years later.

Postmarket monitoring should consider whether:

• the device is being used differently than originally intended;
• clinicians are relying on it more heavily over time;

• new workarounds have developed;

• override rates are changing;

• users are reviewing outputs less frequently;

• the professional group interacting with the device has changed;

• workflow changes have altered the meaning of inputs;

• organizational incentives are encouraging inappropriate reliance; or
 • the device's introduction has changed the baseline against which future performance is
assessed.

This is particularly important for systems that become deeply embedded in routine care.

Once clinicians and organizations reorganize workflows around an AI system, withdrawing or
changing that technology may itself become operationally difficult.

X. Frontline Clinical Disagreement Should Be Treated as Data

One recurring challenge in healthcare technology is that disagreement between a clinician and a
system may be recorded only as an "override."

That description provides limited information.

An override may represent:

• an AI error;

• an unusual patient circumstance;
• incomplete data;

• a clinician error;

• a workflow mismatch;

• an inappropriate threshold;

• an interpretability problem;

• a difference in acceptable risk;

• a documentation artifact; or
• an emerging failure mode.

FDA should encourage manufacturers to develop systems capable of learning from the reason for
clinically significant disagreement rather than merely counting the disagreement.

Where feasible, structured categories for override and correction events could substantially
improve postmarket learning.
Repeated patterns of disagreement among experienced clinicians may reveal problems that
conventional performance metrics fail to identify.
XI. Safety Reporting Must Be Designed for the Reality of Frontline Work

Any reporting framework that depends on frontline clinicians must recognize the practical
conditions under which care is delivered.
A nurse caring for multiple acutely ill patients cannot reasonably be expected to complete a
lengthy technical incident report each time an AI system generates a questionable output.

Reporting systems should therefore minimize the burden of capturing safety events.

Where possible, systems could automatically preserve relevant technical information when a
clinician:

• overrides a recommendation;

• substantially edits AI-generated clinical content;

• rejects a proposed action;

• escalates an AI recommendation;
• flags an output as unsafe or incorrect; or

• identifies a suspected AI-related near miss.

The clinician could then add minimal contextual information.

This would create richer safety data while reducing dependence on voluntary reporting systems
that compete with direct patient-care responsibilities.

XII. Transparency Should Extend to the Work Required to Keep AI Safe

Healthcare organizations evaluating AI technologies should be able to understand not only what
a system can do but also what human infrastructure is necessary for it to operate safely.

Manufacturers should therefore be encouraged to characterize, where applicable:

• expected frequency of human review;

• qualifications required of reviewers;

• known categories of outputs requiring verification;
• circumstances in which users should not rely on the system;

• expected frequency of overrides or corrections identified during validation;

• monitoring requirements;

• escalation responsibilities;

• training requirements; and

• known limitations that increase human workload.

A device that requires substantial expert supervision may still be valuable.
The important point is that the required supervision should be visible.

Clinical labor should not function as an unreported performance subsidy that makes an AI system
appear more autonomous, efficient, or reliable than it is in practice.

XIII. Recommended Principles for FDA Consideration

I respectfully recommend that FDA consider the following principles as it develops its approach
to generative and agentic AI-enabled medical devices:

1. Evaluate the appropriate clinical comparator according to function.

Do not assume that physician performance is the relevant reference point for every AI-enabled
clinical function. Use the healthcare professional or interdisciplinary team that performs the
comparable work in actual practice.
2. Evaluate the AI-enabled clinical workflow when appropriate.

For technologies whose safety or benefit depends heavily on human interaction, assess not only
standalone device performance but also human-AI and clinical-system performance.

3. Measure human rescue.

When clinicians detect, correct, or prevent AI-generated errors from reaching patients, those
events should be recognized as safety-relevant data rather than disappearing because no adverse
outcome occurred.

4. Measure work created as well as work eliminated.

Where productivity, efficiency, or burden reduction is relevant, evaluate the entire affected
workflow and determine whether work was eliminated, reduced, or redistributed.

5. Include nurses in postmarket surveillance.

Nurses and other frontline professionals should have practical mechanisms for reporting
recurrent errors, near misses, overrides, workflow failures, and clinically significant
disagreements with AI systems.

6. Do not transfer manufacturer quality-control responsibilities to clinicians.

Clinical oversight is necessary, but routine human correction of technology should not become
an invisible substitute for adequate product performance and manufacturer surveillance.

7. Define meaningful human oversight for agentic systems.

Oversight should require sufficient information, time, authority, transparency, and ability to
intervene, particularly before high-consequence or irreversible actions.

8. Evaluate the effectiveness of the human safety layer.
If safe operation depends on clinicians catching errors, FDA should consider whether intended
users can reliably detect those errors under realistic clinical conditions.

9. Monitor workflow drift as well as technical performance drift.

The relationship between clinicians and AI systems will evolve after deployment. Postmarket
monitoring should account for changes in reliance, workarounds, staffing, workflow, training,
and organizational behavior.

10. Treat frontline disagreement as a source of regulatory intelligence.

Patterns of overrides, corrections, and disagreement may identify emerging risks before those
risks appear in adverse-event data.

Conclusion
Generative and agentic AI will increasingly move from generating information to participating
directly in clinical work.

As that transition occurs, the distinction between the performance of an AI model and the
performance of the clinical system surrounding it will become increasingly important.

A device may generate highly accurate outputs and still produce poor clinical results if it disrupts
workflow, creates excessive cognitive burden, encourages inappropriate reliance, produces
difficult-to-detect errors, or depends upon substantial unmeasured human correction.

Conversely, a technology with known limitations may still provide substantial benefit when
those limitations are transparent and effectively managed within a well-designed clinical
workflow.

The central regulatory question therefore should not always be:
How well does the AI perform?
In many cases, FDA should also ask:

How well does the healthcare system perform because the AI is there?

For many devices, the appropriate unit of evaluation will be the AI-enabled clinical workflow.

Nurses should be part of that evaluation not simply because nursing is a large healthcare
profession, but because nurses frequently occupy the operational boundary between information
and action, between recommendation and intervention, and between technological failure and
patient harm.
If nurses are required to interpret, verify, correct, escalate, document, or mitigate AI-generated
outputs, that work is part of the real-world performance of the technology.
It should be visible.
It should be measured.

And it should inform how the safety and effectiveness of AI-enabled medical devices are
evaluated throughout their lifecycle.

Thank you for the opportunity to provide these comments.

Respectfully submitted,

Mark Smith, MBA, BSN, RN
Registered Nurse
Submitted in an individual capacity

References

1. U.S. Food and Drug Administration, Center for Devices and Radiological Health.
Considerations for the Regulation of Generative AI-Enabled Medical Devices:
Discussion Paper and Request for Feedback. 2026.

2. U.S. Food and Drug Administration. Applying Human Factors and Usability Engineering
to Medical Devices: Guidance for Industry and Food and Drug Administration Staff.
2026.

3. U.S. Food and Drug Administration, Center for Devices and Radiological Health.
Measuring and Evaluating Artificial Intelligence-Enabled Medical Device Performance
in the Real World: Request for Public Comment. 2025.

4. U.S. Food and Drug Administration. Artificial Intelligence-Enabled Medical Devices.
5. Additional peer-reviewed literature and supporting evidence regarding nursing workflow,
human factors, clinical AI implementation, automation bias, clinician workload, and
healthcare technology evaluation may be submitted as supplementary references where
appropriate.