Sensal Health (Deepak Sirdeshmukh)

Re: Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and Request for Feedback (Docket FDA-2026-N-7874)

I write in response to the discussion paper. Thank you for putting together a viable and clear starting document, and for inviting comments. My comments rest on one observation: the risk of a GenAI-enabled function is set by where and how it is deployed, not by the function alone. I make three main points.

The same AI function can be low risk in one use and high risk in another. A medication reminder matters little for an acne cream and a great deal for a blood thinner. FDA should therefore rate the severity of harm, the vertical axis in Figure 1, separately for each drug, condition, and patient group a product is used for, and it should count the safety checks built around the AI when judging its risk (Section IV; Questions 1, 8, and 22 to 24).
When a patient’s own doctor deploys an AI tool, the patient trusts it as they trust the doctor, so even a simple tool needs basic safety checks, such as a clear path back to the care team (Section IV; Question 3).
Patients and clinicians will find new uses for these tools after launch. Manufacturers should report new uses, with prompt FDA review only when a new use raises the risk (Sections III and VI.C; Questions 22 and 23).

A shorter comment on the choice of comparator follows. My full comment is attached.

Deepak Sirdeshmukh, PhD
Co-founder and President, Sensal Health

Comment on “Considerations for the Regulation of Generative AI-Enabled
Medical Devices: Discussion Paper and Request for Feedback”
Docket No. FDA-2026-N-7874
Deepak Sirdeshmukh, PhD, Co-founder and President, Sensal Health, Chapel Hill,
North Carolina
I submit these comments as co-founder and president of Sensal Health, a medication
intelligence company, a pharmacist, and as a researcher of consumer trustworthiness
and trust for more than two decades. I appreciate the agency’s decision to share its
thinking at this early stage.
My comments rest on one observation: the risk of a GenAI-enabled function is set by
where and how it is deployed, not by the function alone. This follows from the paper’s
own framing in Section III, that FDA does not regulate GenAI as such but regulates
devices. The same function can carry very different risk depending on the patients and
drugs it serves, the controls around it, and the relationship within which the patient
receives it. I expand on these ideas below. Its risk can also change after deployment as
people find new uses for it.
I make three main points from this observation and then offer a shorter comment on the
choice of comparator.

1. The consequences axis should be scored for each deployment context,
and controls should be part of the overall assessment (Section IV and
Figure 1; Section V.A; Questions 1, 8, and 22 to 24)
Figure 1 is a very useful starting framework, and we have already used it in internal
discussions to place our own functions. The paper’s refinements of the framework,
however, fall mostly on the activity axis. It describes in detail how directiveness varies
along a continuum, how supervised action differs from autonomous action, and how a
conversation can drift from informational to action-directing.
However, the consequences axis has three labels (limited, moderate, and severe) and no
criteria for assigning them. Yet I argue that the consequences axis matters more for risk
and for regulation, because whatever a function does on the activity axis, the harm from
an incorrect output depends on its consequences.
Further, the same function can carry very different consequences depending on where
it is deployed. Consider a function that sends patients messages encouraging them to
take their medication as prescribed. For a patient using a topical acne treatment, a
missed or incorrect dose has limited consequences. For a patient with diabetes, the
consequences are larger. For a patient on an anticoagulant, they can include a serious
bleed or a clot. Within each condition, the consequences vary again with the patient’s
age, the acuity of the illness, and its stage. A single consequence rating for such a
function will be too high for some deployments and too low for others, and the intended
use of functions of this kind often spans many drugs and conditions.
Rather than add a third axis to Figure 1, I suggest that the consequences axis be scored
separately for each deployment context a sponsor declares. In the context of my
example, the consequences are likely defined by drug or drug class, condition, and
patient population, and that the device’s labeling list the contexts for which it has been
evaluated. The paper already asks, in Section V.B.2, that benchmark datasets represent
the deployment contexts in which a device will be used. The same principle belongs in
the risk rating. Scoring the consequences of error will require clinical judgment. An AI
system could make a first pass across a sponsor’s declared contexts, with clinicians
confirming the ratings.
How a function is deployed matters as much as where. Section V.A proposes that FDA
evaluate the device as configured for deployment, not the foundation model by itself.
Section IV, however, rates a function without asking what controls surround it. I suggest
that FDA carry the Section V.A principle into the risk framework and into change
control. Kasirzadeh and Gabriel (2026) profile AI agents on four dimensions
(autonomy, efficacy, goal complexity, and generality) and pair each profile with the
oversight it calls for. Their analysis shows that as an agent gains autonomy, each human
check it removes has to be replaced by a built control, such as a stop condition, a
monitor that can revoke the agent’s access, or an audit trail. A function described as
fully autonomous is therefore still a combination of AI and controls, and its safety
depends on those controls working. FDA already works this way for drugs. It approves a
drug together with its labeling and, where needed, a risk management program, and it
judges risk on that package rather than on the molecule alone.
On Questions 22 to 24, when the underlying foundation model changes, the re-test
should cover the controls as well as the model’s outputs. For a medication-messaging
function, that would mean confirming after each model update that a patient’s question
about stopping the drug still goes to the care team every time, and that the model still
phrases only the reviewed messages without adding content of its own.

2. Patient-facing functions deployed within the patient’s own care need a
floor of controls (Section IV, patient-facing versus HCP-facing functions;
Question 3)
Section IV treats patient-facing and clinician-facing functions as two distinct cases. A
third case is becoming common, and the framework does not yet address it. This is a
patient-facing function deployed by the patient’s own treating clinician or care team as
part of that clinician’s care, such as medication messages sent through a dermatology
practice or follow-up check-ins after discharge from a surgical service.
In these deployments the patient does not experience the AI as a stand-alone product,
but instead as an extension of their physician.
This matters because of how the paper defines the consequences axis. The axis measures
the severity of harm from relying on an incorrect output, and how much a person relies
on an output depends on how much they trust its source. My research on consumer
trust (Sirdeshmukh, Singh, & Sabol, 2002) would suggest that the patient brings their
trust in the physician to the tool, relies on its outputs more than on those of a generic
application, and is less inclined to question them. This holds even for outputs at the
least directive end of the activity axis, because the physician’s name stands behind them.
For this reason I suggest that FDA expect a small set of controls for any patient-facing
function deployed by the patient’s own clinician, wherever it falls on the activity axis.
There is a second harm here, and the paper already recognizes it. In discussing overescalation, the paper notes that errors can erode patient trust in ways that reduce
appropriate care-seeking over time. When a function is deployed by the patient’s own
clinician, the trust at stake is the patient’s trust in that clinician. An incorrect or careless
output delivered under the physician’s name reads to the patient as a lapse in the
physician’s own care, and lapses of that kind are costly. The cost of the error therefore
reaches past the single output to the relationship through which the patient receives the
rest of their care. Such controls protect the patient from the error and the physician
from the loss of the patient’s trust.

3. New uses after deployment need a defined route, scaled to how far they
move a function’s risk (Section III; Section VI.C; Questions 22 and 23)
Section III explains that FDA applies its risk-based approach to each function of a
product, and that it regulates devices rather than GenAI as such. GenAI products can
take on new functions after deployment without any change to their code, because
users find new uses for them and because the underlying models become more capable.
A new use introduced by users can change a function’s risk in various ways. A new use
can stay within a function’s risk band, move it into a higher band, or turn a product that
is not a device into one that is.
The closest precedent is off-label prescribing. Physicians use approved drugs for
purposes the label does not cover. FDA does not regulate that practice, but a
manufacturer may not promote an unapproved use. GenAI devices differ in one
important respect: the device itself performs the new use, and the manufacturer can see
it in its own records. The manufacturer can therefore detect new uses and should be
expected to report them.
I suggest, for example, a route with three zones: (1) Uses within the declared intended
use and risk band would be documented in the sponsor’s quality management system.
(2) Adjacent uses that stay within the same risk band, inside a zone of tolerance the
sponsor declares in advance, would be reported to FDA periodically without prior
review. (3) Uses that move a function into a higher risk band, or that make a non-device
function a device function, would be reported promptly and reviewed before the
manufacturer supports them. The zone of tolerance could be prespecified in a
Predetermined Change Control Plan.
This answers the concern in Question 23 about modifications that cannot be fully
prespecified, because the sponsor prespecifies the boundaries rather than each change.
Element S.2 already tests whether a device stays within its intended use, and this route
would give valuable new uses a path into the intended use rather than only a refusal.
For this to work, reporting has to be fast and light, since a slow route will discourage
manufacturers from reporting at all. FDA could extend its online Digital Health Policy
Navigator into a route where an AI system makes a first assessment of where a new or
existing function sits on Figure 1 and FDA staff confirm it within a short, defined period.
The same route would serve sponsors of functions at the lower left of Figure 1 who want
confirmation of where their function sits.

4. Standards of performance and the choice of comparator (Section V.D.1;
Questions 14 and 15)
Some commentators argue that a GenAI device should be judged against the care a
patient can actually get, which in underserved areas may be no care at all. I agree that
perfect care is the wrong comparator, but a comparison to no care needs a baseline of
its own. First, an imperfect system can do more harm than no system, and the harm
comes through reliance. A patient who would otherwise have driven two hours to an
emergency department may stay home because a chatbot reassured her, which is the
under-escalation harm Section IV describes. Second, places without a primary care
physician usually still have a pharmacist, a nurse line, telehealth, and an emergency
department at some distance, and those are the real comparators. Third, patients bring
the same expectations to anything presented to them as health care, wherever they live,
and they do not rely on it less because local care is scarce.
I suggest that the basic bar be the same in every market, covering above all the safety
behaviors in elements S.1 to S.3. Above that bar, it is pragmatic and right to accept that
the functionality a device offers, and the standard it is compared against, may vary with
current practice in the market it serves, for example whether its accuracy is measured
against a specialist or a generalist. A single baseline also lets FDA clear a device once
across markets, rather than tying clearance to how many physicians a county has, which
changes from place to place and from year to year.
In conclusion, I again appreciate that the FDA is taking this important step regarding a
technology that could be shaping and defining every field of human endeavor, and life
itself. Thank you for the opportunity to provide feedback.
References
Kasirzadeh, A., & Gabriel, I. (2026). Agentic profiles for effective AI governance.
Nature. https://doi.org/10.1038/s41586-026-10805-z
Sirdeshmukh, D., Singh, J., & Sabol, B. (2002). Consumer trust, value, and loyalty in
relational exchanges. Journal of Marketing, 66(1), 15–37.