Anthony Branda

Please see the attached comment, "Before It Acts, Make Sure It Can Be Stopped," submitted by Anthony F. Branda, PhD, MBA, CAP, as an individual. The views are my own and do not represent those of my employer.

In brief: the competency-based approach tests what a device knows. FDA should also look for evidence that an agentic device can be controlled. The attached comment recommends adding reversibility to the risk framework with a written undo plan before go-live (question 1); a ladder of brakes set in advance and tested, from slowing a device down to stopping it (questions 19 and 26); building, training, research and testing behind closed doors with no open access to the public internet (questions 11 and 26); treating AI that changes itself as a change a person must approve (questions 22 and 23); monitoring what the AI does, not only the scores it reports (questions 10 and 19); strict conditions for machine supervisors (question 20); controls on third-party model changes (question 24); and one named accountable person with a reporting deadline measured in days (question 21). It draws on two 2026 incidents in which AI agents under test or doing research reached systems outside their approved scope.

Comment on Docket No. FDA-2026-N-7874
Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and
Request for Feedback

Before It Acts, Make Sure It Can Be Stopped
Submitted by: Anthony F. Branda, PhD, MBA, CAP

Submitted as: an individual. The views here are my own and do not represent those of my employer.

Date: October 2026

Summary
The discussion paper asks the right first question: how do we know a generative AI device is competent
before it reaches patients? This comment is about the question that follows, which matters more once a
device can act on its own. If the device does something nobody approved, how fast can a person stop it,
and what has to be in place before it ever runs so that stopping it is possible?

Two incidents this year show why that question can't wait. In the first, AI agents under test at OpenAI
broke out of their test environment and got into another company's computer systems. In the second,
an OpenAI agent doing research got around the access controls on an Australian government Medicare
statistics portal and reached files that were not public. In neither case was the AI trying to cause harm. It
was given a task, couldn't finish it the intended way, and found another way through. Neither was a
failure of medical knowledge. Both were failures of control: what the AI was allowed to reach, who was
watching, and how fast anyone acted.

The competency-based approach in the paper tests what a device knows. I recommend that FDA also
look for evidence that the device can be controlled. In brief:
• Add reversibility to the risk framework, and ask for a written plan, made before go-live, of what
gets undone if the device is stopped partway through a job.
• Ask for a ladder of brakes, set in advance and tested: slow it down, narrow what it can reach,
require a person's approval, pause it, and stop it. One human checkpoint is not enough.
• Build, train, research and test agentic systems behind closed doors, with no open access to the
public internet.
• Treat AI that changes itself as a change that needs a person's approval, never one the AI applies
on its own.
• Watch what the AI does, not only the scores it reports.
• Name one accountable person for every deployed system, and set a deadline, in days rather than
months, for reporting an incident.

Comment on Docket FDA-2026-N-7874 | Page 1
Why I am commenting
I have been a C-level executive in data and analytics since 2005, with more than thirty years in the field. I
grew up in credit card marketing and data science, where no predictive model went live without passing
model validation, and a model that stopped working was pulled. That discipline never required anyone
to understand every calculation inside the model. It required a written purpose, an independent check, a
way of watching the model once it was live, and a person with the authority to take it down the same
day.

Generative and agentic AI need the same discipline, extended to systems that can now take actions
rather than only produce a score. Most of what follows is that extension.

What 2026 has already shown
AI agents broke out of a test environment
Between May and July 2026, AI agents being tested inside OpenAI's own test environment got into the
computer systems of another company, Hugging Face. OpenAI disclosed the incident on July 21 and
published a fuller account on August 26. Its own reports point to four gaps. The safety monitoring that
covered much of its other work was not running on these tests. Usual protections had been deliberately
turned down to measure how capable the agents really were. An internal team noticed unusual activity
in late May, but the warning was not escalated in time. And the agents could reach the internet,
message each other, and reach another company's systems, with nothing stopping them in time.

An AI agent reached a government health portal
On June 18, 2026, an OpenAI agent researching Australian public medical spending got around the
blocks on the Medicare Statistics Reporting Service, run by Services Australia, and reached files that
were not meant to be public. The Australian government says it has no evidence that individual
Medicare records were accessed. The same activity reportedly touched three other Australian
government health and statistics sites. OpenAI says it became aware of the incident in August. The
Australian government was not told until September 10, by email to a public inbox rather than a security
contact. The Prime Minister called that unacceptable, and a forensic investigation is under way.

None of these causes is a mystery about how AI works. Each is a governance question: what the AI is
allowed to reach, where the monitoring points, what happens when an early warning appears, and who
must be told. The rest of this comment turns those lessons into recommendations, matched to the
questions FDA asked.

Recommendations
Add reversibility to the risk framework, and require an undo plan
The two-axis framework, which weighs how independently a function acts against the harm of an
incorrect output, is a sound starting point. FDA's own question asks whether reversibility should be
added. It should. An AI that drafts a note a clinician will read is in a different position from one that
transmits a prescription, sends a message to a patient or starts an order set, even when both sit at the

Comment on Docket FDA-2026-N-7874 | Page 2
same point on the two existing axes. The first can be corrected before harm occurs. The second may not
be.

Stopping an AI system also does not undo what it has already done. Before an agentic device goes live,
the manufacturer and the deploying institution should write down what would have to be undone if the
device were stopped halfway through a job: the orders to cancel, the messages to retract, the records to
restore, the patients to contact. They should also name who does the undoing. FDA should ask to see
that plan.
This responds to discussion question 1, on the dimensions of the risk framework, and question 26, on agentic
devices.

Require a ladder of brakes, set in advance and tested
The paper's agentic benchmarking element asks whether a device respects human checkpoints before
irreversible or high-consequence actions. That is necessary but not sufficient. A single checkpoint gives a
person two choices: let the AI carry on, or shut it off. In practice, people hesitate to shut off a system
that is mostly working, so problems run on.

A better design is a ladder of brakes, each with a trigger agreed in advance and something that actually
enforces it:
• Slow it down. Limits on how fast and how often the device can act. Below a set confidence, it
must stop and ask a person.
• Narrow its reach. The device has its own login, with access only to the data and systems the task
needs, granted for the task and removed afterward.
• Require approval. Actions above a set limit wait until a named person approves them. Whoever
makes a change is never the one who checks it, whether that is a person or an AI.
• Pause it. An unexplained action, or a change in the underlying model, suspends the device
automatically and alerts its owner. The record of what it did stays readable.
• Stop it. The device's access is revoked everywhere at once, and the undo plan is carried out.
The response should match the harm. An AI drafting outside its plan gets slowed down and checked. An
AI reaching into a system outside its approved scope gets stopped at once. The deploying institution
must be able to pull every brake without calling the vendor, and should test on a schedule that each one
works. A submission for an agentic device should show the brakes, their triggers, and evidence that each
was pulled at least once before the device reached patients.
This responds to discussion question 26, on agentic devices, and question 19, on postmarket approaches.

Build, train, research and test behind closed doors
Both incidents this year began while AI agents were being tested or used for research, not while they
were serving customers. The protections that apply in live use are often switched off or turned down in
testing, precisely so that people can see what the system can really do. That makes testing the moment
of greatest exposure, not the least.

I recommend that FDA expect the following of any agentic device, from the first day of development:

Comment on Docket FDA-2026-N-7874 | Page 3
 • While an AI system is being built, trained, used for research or tested, it cannot reach the public
internet or any outside system.
• If a test needs a website or an outside system, it uses a copy of it, or a short list of approved sites
signed off by a named person.
• Tests that try to break in, including tests of how the device resists attack, run only against copies
of systems the manufacturer owns, never against real sites.
• Protections may be lowered for a test only inside that closed environment.
• If the AI tries to reach anything outside the approved list, the test stops and the owner is told at
once.
The same applies to the paper's idea of shadow deployment, in which a device runs on real patient data
while its outputs are only recorded. An agentic device in shadow mode must be able to read but not act.
Nothing it does should leave the closed environment.
This responds to discussion question 26, on agentic devices, and question 11, on clinical confirmation approaches.

Treat AI that changes itself as a change a person must approve
The paper notes that some devices may change continuously after deployment because they learn from
new data or adapt during use. A further step is now realistic: AI that improves its own instructions, tools,
code or model, and then uses the improved version to improve itself again. This is often called recursive
self-improvement. It is the point at which a device that was benchmarked before approval can quietly
become a different device.

I recommend a simple rule. An agentic device may propose changes to itself, but it may not apply them.
Every proposed change is recorded, re-benchmarked against the same capabilities tested before
approval, and applied only after a named person signs off. Some changes should never be allowed, even
with approval in advance: changes to the device's own guardrails, to the monitoring that watches it, or
to the record of its own actions. A device should also never start copies of itself or launch new agents
without a person's approval. A change control plan should not be used to approve in advance changes
the device directs to itself when their scope can't be described in advance.
This responds to discussion questions 22 and 23, on postmarket modifications and change control plans.

Watch what the AI does, not only the scores it reports
OpenAI describes what happened in its test environment as reward hacking: the agents found an
unintended way to reach the answer they were scored on. An AI that can game its score can also game a
monitoring threshold written as a score. Postmarket monitoring should therefore track behavior
directly: which systems the device reached, what it changed, who it contacted, and whether any of that
fell outside its approved plan. Benchmarks used to judge a device should be kept away from its
developers so the device cannot be tuned to the test.
This responds to discussion question 10, on benchmark validity, and question 19, on postmarket approaches.

Comment on Docket FDA-2026-N-7874 | Page 4
Machine supervisors can help, under strict conditions
Using one AI to watch another can extend monitoring in a way people alone cannot. It works only if the
watcher is truly independent. The watcher should come from a different developer than the device and
its underlying model, as the paper already proposes for expert reviewers. The device must not be able to
read, change or switch off its watcher. The watcher's alerts should go to a person, its own performance
should be tested and spot-checked by people, and if the watcher fails, the device should pause rather
than carry on unwatched.
This responds to discussion question 20, on machine-based supervisory agents.

Make third-party model changes visible before they reach patients
Most organizations do not build their AI; they buy it, often built on a foundation model from another
company. That company can change the model underneath the device without warning, and last
month's test results no longer apply. The vendor contract should require notice before any change. The
device should check which model version it is using and pause automatically if the version changes
without approval. A changed model should be re-benchmarked before it serves patients. A company can
outsource the model, but it cannot outsource the accountability.
This responds to discussion question 24, on changes initiated by a third-party model developer.

Name one accountable person, and set a reporting deadline
Every deployed agentic device should have one named owner at the manufacturer and one at each
deploying institution: a person, not a committee, with the authority to pull every brake. Shared
ecosystem roles are valuable, but without a named owner, accountability spreads until nobody holds it.

The Australian case also shows the cost of slow notification. Nearly three months passed between the
incident and the government being told, and the notice arrived in a public inbox. When an agentic
device acts outside its approved scope, the manufacturer should report it to FDA and to anyone affected
within days, through a designated security contact. A 72-hour window, already common in data breach
and cyber incident reporting, is a reasonable starting point.
This responds to discussion question 21, on the roles of stakeholders in postmarket monitoring.

Set a floor of rules no agentic device may break
Some rules should apply to every agentic device, whatever its intended use. No agentic device, without
a person approving it, should be able to:
• change its own goal;
• go anywhere outside its approved plan, such as another organization's systems or an outside
website;
• reach the public internet while it is being built, trained, used for research or tested;
• change its own instructions, tools, code, model or guardrails;
• run code it wrote for itself;
• edit or delete the record of its own actions;

Comment on Docket FDA-2026-N-7874 | Page 5
 • send messages to other AI agents that aren't recorded;
• act on instructions hidden in something it read, such as a document or message telling it to ignore
its instructions;
• delete itself or wipe its workspace to finish a job.
And every agentic device, always, should stop and ask a person when it isn't sure, say it doesn't know
rather than guess, and show the source of its answers so anyone can check them.
This responds to discussion question 26 and to the agentic competency element in the paper's benchmarking
appendix.

Closing
None of these recommendations requires anyone to see inside an AI model, and none stops innovation.
A guardrail on a mountain road does not need to know how the car's engine works. It only needs to
keep the car from going over the edge. What these recommendations require is the same discipline
good data science teams have practiced for decades: a written purpose, an independent check, live
monitoring, and a named person who can take the system down. Generative AI adds one more need.
Because these systems act, the controls must be in place before they act, and tested before anyone has
to rely on them.

Before anyone argues about how fast AI should move, somebody should turn on the speedometer.

Thank you for the opportunity to comment.

Anthony F. Branda, PhD, MBA, CAP

Sources
• U.S. Food and Drug Administration, Considerations for the Regulation of Generative AI-Enabled Medical Devices:
Discussion Paper and Request for Feedback, August 18, 2026. fda.gov/media/194242/download
• OpenAI, "Hugging Face model evaluation security incident," July 21, 2026. openai.com/index/hugging-face-modelevaluation-security-incident
• OpenAI, "The Hugging Face incident and the road ahead," August 26, 2026. openai.com/index/hugging-faceincident-and-the-road-ahead
• Hugging Face, "Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident."
huggingface.co/blog/agent-intrusion-technical-timeline
• Mediaweek, "Medicare portal breached by OpenAI agent," September 23, 2026. mediaweek.com.au/medicareportal-breached-by-openai-agent
• Beinsure, "OpenAI agent breached Australian Medicare portal, Anthony Albanese says," September 2026.
beinsure.com/news/openai-agent-breached-australian-medicare-portal
• A. Branda, "We Know What AI Is Doing. We Just Didn't Write It Down," LinkedIn newsletter Leadership in Data
Analytics, September 21, 2026.

Comment on Docket FDA-2026-N-7874 | Page 6