← All 530 filings

DeNovo Innovations Group (Gennaro Maida)

IndustryConsultantFiled October 19, 20265,019 words · 1 attachmentFDA-2026-N-7874-0386
LinkedInX

Themes it raises

12 of the 21 themes in the docket, each with the passage we counted, verbatim.
What makes a function high riskFDA Q1, Q2, Q5
“Reversibility belongs on the consequences axis, with a condition: it should be demonstrated, not asserted. A rollback that has never been exercised is not a safeguard.”
Whether benchmark results prove anythingFDA Q9, Q10, Q16
“Retire or replenish cases based on how often they have been optimized against. A set that has been tuned against is no longer a test.”
Watching the device after it shipsFDA Q19, Q20
“Before relying on one as a risk control, specify what it monitors, what it can observe, what it cannot, how fast it responds, and what authority it has.”
Controlling a device that keeps changingFDA Q22, Q23, Q24, Q25
“Manufacturers should seek stable version identifiers, advance notice of safety-relevant changes and retirements, access to an evaluation environment, and an achievable migration window.”
Devices that plan and take actionsFDA Q26
“A model may propose an action. It should not be the thing that grants itself permission to perform it.”
Whether human oversight is real oversightFDA Q3, Q4, Q14, Q20, Q21, Q26
“Run simulations with plausible but wrong recommendations, correct ones, and appropriate deferrals, under representative workload and time pressure, and measure whether users catch consequential errors and intervene in time.”
Records that let investigators reconstruct an eventFDA Q19, Q21, Q24, Q26
“Retain enough to reconstruct the input, output, action, observation, and decision, with patient data minimized and access controlled.”
What the rules cost sponsors and the marketNot asked by the FDA
“third-party review should concentrate on high-consequence claims, with reusable test specifications and recognition of equivalent evidence, so that it does not become a toll booth for smaller sponsors.”
Proving the device works in real careFDA Q11, Q12, Q13, Q14, Q15
“Synthetic and adversarial cases are good for sensitivity to specified hazards; they are not estimates of real-world prevalence.”
Security, dependencies and what happens when they failFDA Q1, Q9, Q24
“Test the supervisor with representative good and bad cases, including adversarial retrieved content, delayed or missing events, unavailable tools, corrupted outputs, and loss of access to the monitored system.”
How this fits rules that already existFDA Q8, Q9, Q16, Q25
“An evaluator-integrity record, an agent-action test supplement where relevant, and a change-to-evidence impact assessment could live in existing quality-system records and test reports and be supplied by reference.”
Who is accountable when something goes wrongFDA Q21
“Responsibilities should connect manufacturers and deployers without making institutions responsible for validating a supplier's model.”

FDA questions it names

Questions this filing names by number.

Q1 · The two-axis risk frameworkQ10 · Benchmark contamination and saturationQ16 · Independent third partiesQ20 · Machine-based supervisory agentsQ22 · Re-benchmarking after a modificationQ24 · Third-party foundation model changesQ26 · Agentic devices

Machine-assisted draft, pending human review. The source text and highlighted passages appear below. Read the filing on regulations.gov ↗

The comment as filed

Comment submitted on regulations.gov. Passages we counted are highlighted.

Docket No. FDA-2026-N-7874 — Comment on "Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and Request for Feedback."

Submitted by Gennaro Maida, MS, CEO, DeNovo Innovations Group. The full comment is in the attached PDF, "Evidence Integrity and Action-Level Assurance for Generative AI-Enabled Medical Devices." The attachment is the complete and authoritative text of this comment; the summary below is provided for convenience. We have no product before the Agency.

The discussion paper gets the hard things right. It evaluates the deployed device rather than the bare model, it treats clinical confirmation as separate from benchmarking, and it asks who adjudicates and how independent they are. These comments are refinements to that approach, not a replacement for it.

Three questions get collapsed into one score far too often: did the device produce an acceptable output, did the evaluation actually observe the device, and is the device permitted to act on what it produced. They are different questions with different evidence. Four recommendations keep them separate.

1. Evaluate the evaluator. Every consequential conclusion in a submission should trace to the configuration tested, the cases that actually ran, what the test could not see, and who accepted the result. Missing evidence should stay visibly missing rather than disappearing into a pass rate.

2. Treat time as a property of evidence. A source, a retrieved passage, or a benchmark can be authentic and still wrong for the date. Submissions should show that the corpus state at the moment of any output can be reconstructed.

3. Separate recommendation from permission. A probabilistic model may propose an action. It should never be the thing that grants itself permission to perform it. For agentic devices, that boundary should be a test object, not a design claim.

4. Count oversight only when it has been tested. An approval button is a safeguard to the extent that representative users, under representative conditions, catch the errors it is there to catch.

Everything here should scale with intended use, autonomy, severity and reversibility of harm, and the time available for a human to intervene.

The attachment responds to seven discussion questions:

Q1, additional risk dimensions — keep two axes and treat the dimensions the Agency names as modifiers.

Q10, benchmark validity, contamination, and evaluator integrity — where a sponsor cannot know a third-party model’s training data, the burden shifts to showing that gating material never entered development, tuning, prompt selection, or adjudication under the sponsor’s control.

Q16, independent third parties — define independence by function rather than by organization chart: who generated the cases, who challenged them, who collected the observations, and who accepted the conclusion.

Q20, machine-based supervisory agents — hold a supervisory agent to the same standard as the device, and specify before relying on it what it monitors, what it can and cannot observe, how fast it responds, and what authority it has.

Q22, scaling reassessment to the change — maintain a versioned dependency record connecting each clinically important competency and hazard to the components that support it, recording the state actually tested.

Q24, third-party foundation model changes — contractual notice and technical observability both matter, and neither is enough alone.

Q26, agentic devices and the action boundary — make the action boundary an explicit test object.

Attachment

Attachment, text extracted from the filed document. Passages we counted are highlighted.

PUBLIC COMMENT
FDA Docket No. FDA-2026-N-7874 | CDRH discussion paper, August 2026

EVIDENCE INTEGRITY · ACTION-LEVEL ASSURANCE

Evidence Integrity and
Action-Level Assurance for
Generative AI-Enabled
Medical Devices
A Public Comment on Considerations for the Regulation of Generative AI-Enabled
Medical Devices
Four positions: evaluate the evaluator, date the evidence, enforce permission outside the model, test
the oversight

E VA L U AT E D AT E PERMIT TEST

Gennaro Maida, MS – CEO, DeNovo Innovations Group 19 OCTOBER 2026
ORCID 0009-0000-3065-2550
SUBMISSION EDITION v1.3

Prepared for submission to FDA Docket No. FDA-2026-N-7874 via Regulations.gov.
PUBLIC COMMENT | FDA-2026-N-7874

Submission record
Author: Gennaro Maida, MS | CEO, DeNovo Innovations Group

ORCID: 0009-0000-3065-2550

Edition: Public comment, submission edition v1.3 | 19 October 2026

Submission destination: U.S. Food and Drug Administration, Center for Devices and Radiological Health,
Docket No. FDA-2026-N-7874 (Regulations.gov). Comment period closes October 19, 2026.

Subject: Evidence Integrity and Action-Level Assurance for Generative AI-Enabled Medical Devices: A Public
Comment on Considerations for the Regulation of Generative AI-Enabled Medical Devices (Discussion Paper
and Request for Feedback, August 2026)

Responds to: Discussion questions 1, 10, 16, 20, 22, 24, and 26.

Source text: Markdown, version 1.3, frozen 19 October 2026, SHA-256
6d122a7fbfbbac4d06515754a124a16db641deffb786d3882bc2bd616c409dcf. This PDF is a format-only rendering
of that text. The body, from the title through the references, is word-for-word identical to the frozen source; the
cover, this page, and the following page are front matter.

Suggested citation: Maida, G. "Evidence Integrity and Action-Level Assurance for Generative AI-Enabled
Medical Devices." Public comment to the U.S. Food and Drug Administration, Docket No. FDA-2026-N-7874,
October 2026.

Release and scope: This public comment is filed on behalf of DeNovo Innovations Group. The views are the
firm's and the author's, not those of any client, and we have no product before the Agency. It is not an FDA
publication, endorsement, or regulatory determination.

DeNovo Innovations Group | 19 October 2026 | 2
PUBLIC COMMENT | FDA-2026-N-7874

Purpose and requested FDA action
To the Center for Devices and Radiological Health: Thank you for the opportunity to comment. The discussion
paper gets the hard things right, and this comment refines its approach rather than replacing it. It answers seven
of the twenty-six discussion questions and asks the Agency to make four things explicit as the competencybased approach develops: that the evaluation itself is a tested object, that evidence carries a date, that
permission to act is enforced outside the model, and that human oversight is demonstrated rather than
presumed.

Table 1. The four positions and where they are argued

Position Questions What the comment asks for

1 Evaluate the evaluator 10, 16, 20 A run-level record for every reported result; PASS / FAIL /
UNKNOWN kept apart from coverage; LLM judges and
supervisory agents evaluated as instruments;
independence defined by function, with shared
dependencies disclosed.

2 Treat time as a property of 1, 22, 24 Traceability that carries a date; three clocks kept distinct;
evidence retrieval corpora handled as change-control components;
pinned third-party model versions with predefined
responses to silent updates.

3 Separate recommendation 26 Authorization enforced outside the generating model and
from permission checked at execution; the action boundary tested through
real enforcement paths, with resulting state verified
independently.

4 Count oversight only when it 1, 20, 26 Reversibility and downstream safeguards demonstrated,
has been tested not asserted; human checkpoints exercised under
representative workload before they count as controls.

Everything proposed scales with intended use, autonomy, severity and reversibility of harm, and the time
available for a human to intervene. The limitations of the evidence behind these positions are stated in one
place, the Limitations section, and nowhere else.

DeNovo Innovations Group | 19 October 2026 | 3
PUBLIC COMMENT | FDA-2026-N-7874

Evidence Integrity and Action-Level Assurance for Generative AIEnabled Medical Devices
Docket FDA-2026-N-7874 | Comment on Considerations for the Regulation of Generative AI-Enabled Medical
Devices (August 2026) [1]
Gennaro Maida, MS | CEO, DeNovo Innovations Group

Who is commenting
I build the systems this paper is about: retrieval-augmented and agentic AI for healthcare organizations, and the
security and compliance architecture around them. My training is in biomedical engineering and clinical
research, clinical medicine, infrastructure and applied AI architecture, and the positions below come from
building, breaking, and fixing these systems. Earlier this month I published a public comment on NIST AI
200-2 on evaluator integrity and incomplete evidence; several recommendations below carry that work into the
device context. [2]

I am commenting on behalf of DeNovo Innovations Group, my own firm. These views are the firm's and mine,
not those of any client, and we have no product before the Agency.

The discussion paper gets the hard things right. It evaluates the deployed device rather than the bare model, it
treats clinical confirmation as separate from benchmarking, and it asks who adjudicates and how independent
they are. [1] My comments are refinements to that approach, not a replacement for it. I respond to Questions 1,
10, 16, 20, 22, 24, and 26.

Summary
Three questions get collapsed into one score far too often: did the device produce an acceptable output, did the
evaluation actually observe the device, and is the device permitted to act on what it produced. They are
different questions with different evidence. My recommendations keep them separate.

1. Evaluate the evaluator. Every consequential conclusion in a submission should trace to the
configuration tested, the cases that actually ran, what the test could not see, and who accepted
the result. Missing evidence should stay visibly missing rather than disappearing into a pass
rate.
2. Treat time as a property of evidence. A source, a retrieved passage, or a benchmark can be
authentic and still wrong for the date. Submissions should show that the corpus state at the
moment of any output can be reconstructed.
3. Separate recommendation from permission. A probabilistic model may propose an action. It
should never be the thing that grants itself permission to perform it. For agentic devices, that
boundary should be a test object, not a design claim.
4. Count oversight only when it has been tested. An approval button is a safeguard to the extent
that representative users, under representative conditions, catch the errors it is there to catch.

DeNovo Innovations Group | 19 October 2026 | 4
PUBLIC COMMENT | FDA-2026-N-7874

Everything here should scale with intended use, autonomy, severity and reversibility of harm, and the time
available for a human to intervene.

Question 1 – Additional risk dimensions
The two-axis framework is a good heuristic. I would keep two axes and treat the dimensions the Agency names
as modifiers, with two notes. [1]

Reversibility belongs on the consequences axis, with a condition: it should be demonstrated, not asserted. A
rollback that has never been exercised is not a safeguard.
The same holds for downstream safeguards. A
human checkpoint or an institutional stop control moves a function down the risk gradient only to the extent the
submission shows it works under the conditions of use (see Question 26).

Traceability to primary sources should carry a date. "Traceable" should mean traceable to a source that was
in effect for the clinical date in question. A citation to guidance that had already been superseded is a
traceability failure even when the quoted passage is authentic. In my own policy-retrieval work I modeled each
manual section's revisions as dated nodes, each release linked to the revision that last changed its content, and
keyed them on publication date; a reimbursement rule from December can be superseded in January, and
answering a query means knowing when the policy was in force, not only what it said. I also defined effectivedate and supersession fields in that schema and never populated them. A schema field is not a control until
something reads it. Appendix A, element S.3, already treats presenting outdated information with false
confidence as a safety failure; the risk framework should recognize the same thing. [1]

Question 10 – Benchmark validity, contamination, and evaluator integrity
Contamination. The Agency is right to flag it. [1] For devices built on third-party foundation models, the
sponsor cannot know what was in the training data. So the burden shifts: show that gating material never
entered development, tuning, prompt selection, or adjudication under the sponsor's control, and say plainly
what cannot be known about upstream training data. Concretely:

• Hold back a sequestered case set, assembled after the model and configuration freeze, that neither the
sponsor nor the foundation model developer has seen. Report it separately from public and sponsordeveloped benchmarks.
• Disclose the exposure history of every benchmark asset: public or private; used in development, tuning,
prompt selection, or adjudication; and who had access when.
• Retire or replenish cases based on how often they have been optimized against. A set that has been tuned
against is no longer a test.

• Evaluate an LLM judge as an instrument. Test it against qualified reference adjudication on positive,
negative, and ambiguous controls, and probe for correlated errors when the judge shares a model family or
training sources with the device. Section V.B.2 says independence considerations apply when the
adjudicator is an LLM; I would add that "different vendor" is not independence when both models drew on
the same public corpora. [1]

I think of this as an epistemic clean room: evaluation material never enters the development environment, and
the submission records custodianship, access, release history, and reuse so a reviewer can inspect the
separation. Held-out sets and rotation are established practice; what I am adding is the requirement to document

DeNovo Innovations Group | 19 October 2026 | 5
PUBLIC COMMENT | FDA-2026-N-7874

the separation. Sponsor-developed benchmarks remain valuable for specialized hazards and should be used,
with their exposure history disclosed and an external challenge set alongside them.

Construct validity. Protecting a benchmark from contamination is necessary and not sufficient. The sponsor
should map each benchmark element to a clinically meaningful failure mode, the target population and setting,
the reference standard, the prespecified acceptance criterion, and the consequences of error. Compare
benchmark findings with independent realistic cases and the chosen clinical-confirmation evidence, and
investigate mismatches rather than averaging them away. Measure severe-error rates, appropriate abstention
and escalation, and subgroup behavior. Synthetic and adversarial cases are good for sensitivity to specified
hazards; they are not estimates of real-world prevalence.
A sequestered but clinically unrepresentative
benchmark is still a poor gate.

Evaluator integrity. A benchmark score is only as good as the pipeline that produced it, and that pipeline is
itself software that fails. A dated account in my own engineering repository (September 2026) records test steps
that reported success after collecting none of the intended tests; the original run output has not been recovered.
On 16 October 2026 I reproduced the failure mode locally on the current code: a test command whose name
filter matched nothing exited with status 0 after executing none of its four collected tests, and a test step whose
required service was deliberately unreachable skipped every case and also exited 0. These are local results at a
pinned revision, not recovered CI outcomes. In one of my own retrieval systems, fifty-seven undated warm-up
queries each returned a clean 200 with no revisions found, and nothing flagged the missing date; for the two
dated queries that did run, the logs record that date selection executed but not which date was asked or what
came back. The generalizable failure mode is a green status with absent, zero, or incomplete effective coverage.
That is an evaluation failure, not a device failure, and treating it as a pass is worse than either.

I recommend a short run-level record for every reported result: the device and evaluator versions; data, rubric,
and configuration; what the test intended to observe and what it actually observed; planned versus executed
sample counts; execution failures and exclusions; and the derivation from raw observations to the reported
number. This is the discipline described in my NIST comment, adapted to device submissions. [2] NIST's
current work on evaluation probes for agentic systems is adjacent; it is research rather than a validation
standard. [3]

Two practices make this record meaningful. First, run known-good and deliberately deficient cases through the
same path as the assessed cases — in a device context, for example, a fabricated citation, a valid but superseded
source, a missing modality, a wrong unit, a tool result bound to a different patient. If the evaluator cannot flag
the planted defect, its passes mean little. Second, keep the criterion decision and coverage apart. A criterion is
PASS when applicable evidence establishes it, FAIL when evidence shows a violation, and UNKNOWN when
no supported determination can be made, with the reason recorded. I record coverage separately as
COMPLETE or PARTIAL. A PARTIAL run can support a narrowly scoped PASS or FAIL; it cannot be
presented as comprehensive, and UNKNOWN is never a pass by exclusion. [2]

For grounded outputs, assess four things separately: does the source exist, is it authoritative, was it in effect at
the time, and does it support the specific claim. A model's calibrated confidence does not substitute for any of
them. Repeated stochastic runs should be used to characterize variability in safety-critical behavior; exact
textual repetition is not the goal.

DeNovo Innovations Group | 19 October 2026 | 6
PUBLIC COMMENT | FDA-2026-N-7874

Question 16 – Independent third parties
Third parties can usefully hold sequestered case sets, adjudicate, audit evaluators, and replicate targeted results.
I would define independence by function rather than by organization chart: who generated the cases, who
challenged them, who collected the observations, and who accepted the conclusion. Those four roles need to be
named, any overlap between them documented, and acceptance kept separately accountable. They do not need
four companies. [2]

Shared dependencies matter more than shared letterhead. Disclose model families, evaluation data where
known, prompts, retrieval sources, and infrastructure. Two parties running the same model on the same corpus
are not independent, whatever the contract says.

Two cautions. Independence should not require releasing patient data or model weights; witnessed execution
and bounded adjudication are workable alternatives when full replication is impractical. And third-party review
should concentrate on high-consequence claims, with reusable test specifications and recognition of equivalent
evidence, so that it does not become a toll booth for smaller sponsors.
Third parties should supplement
manufacturer responsibility and FDA's decision, not replace either. [1]

Question 20 – Machine-based supervisory agents
Supervisory agents can help, and I would hold them to the same standard as the device. Before relying on one
as a risk control, specify what it monitors, what it can observe, what it cannot, how fast it responds, and what
authority it has.
Skovos made the point from the deployer's side: a supervisor that can only observe adds
reporting without adding safety. [6] I agree, and would add that a supervisor that can act is now itself an agent
with the same evaluation obligations as the device.

Test the supervisor with representative good and bad cases, including adversarial retrieved content, delayed or
missing events, unavailable tools, corrupted outputs, and loss of access to the monitored system.
Measure what
it missed, what it falsely flagged, whether the alert was delivered, and how long until someone effective acted.

Shared dependencies deserve explicit challenge. A device and its supervisor running on the same model or the
same corpus can fail together, and two models reading the same source can agree, confidently, on the same
wrong answer. I use a second model to challenge the first in my own work; it does not make the pair
independent. Review should include source-grounded negative controls and observations that neither the
generating nor the supervising model produced. Role separation helps. It is not independence.

A monitoring report should distinguish four states: no adverse events observed, no eligible cases encountered,
observation incomplete, and monitoring unavailable. Each needs a predefined response. Immediate shutdown is
not always the safe one; a device embedded in care has to fail toward continuity. A bounded assurance
argument is better than an unending chain of agents supervising agents.

Question 22 – Scaling reassessment to the change
I recommend a versioned dependency record that connects each clinically important competency and hazard to
the components that support it: foundation model, adapters, retrieval corpus, prompts, tools, authorization
policy, interfaces, input modalities, and the evaluator itself. Record the state actually tested, not a product
name.

DeNovo Innovations Group | 19 October 2026 | 7
PUBLIC COMMENT | FDA-2026-N-7874

Three clocks matter, and they are not the same: when a source was in effect for a clinical context, when the
system ingested or observed a particular version or output, and when the evidence was assessed and the
criterion decided. Corrections arrive later and sometimes apply retrospectively. Preserve prior findings and
dated reassessments; later evidence amends the record rather than silently rewriting it. [2]

Reassessment should follow plausible impact, not the size of the diff. A one-line prompt edit or a changed
retrieval priority can move a high-consequence behavior; a demonstrably isolated change may justify targeted
regression only. The sponsor should say which claims a change touches, which evidence still applies, and why
omitted tests are unnecessary. Accumulated small changes deserve the same analysis.

Retrieval corpora are a change-control component, and time is the axis that matters. In my policyretrieval work I carried section revision identifiers and release dates as explicit fields from ingestion through
the graph, the vector index, and the citations, and selected the release by date for each dated query, because a
retrieved passage can be verbatim correct and wrong for the decision date. The same system recorded when
every document was loaded and never read that timestamp, and its citation expansion step pulled neighboring
passages without restricting them to the resolved revision. The date selection could run exactly as designed and
the answer could still carry text from a release it had not selected. Every decision-relevant source should carry
its effective interval and the date the system indexed it; every material replacement should be recorded as a
typed relation (supersedes, amends, retires); and for safety-relevant outputs the submission should show what
the system could retrieve and what it actually used, through every step that touches the answer. A corpus refresh
should be treated as a device change. It can be a routine one, under defined sources, inclusion criteria, hazards,
acceptance limits, regression tests, and rollback, but it is a change and I would have it get an impact
assessment.

The August 2025 PCCP guidance distinguishes modifications specified in an authorized plan and implemented
under its protocol from everything else. [4] That distinction should survive any adaptation to generative
devices. Bounded corpus refreshes and specified adapter updates are candidates for a PCCP where the plan
specifies them, and sponsors should write them in. [4] Anything outside the authorized plan, whether expanded
clinical scope, a new action capability, or adaptation that cannot be prespecified, gets its own assessment before
it ships. [4] Changes to the evaluator should also trigger a check that the device did not "improve" because the
ruler changed.

Question 24 – Third-party foundation model changes
Contractual notice and technical observability both matter, and neither is enough alone. Manufacturers should
seek stable version identifiers, advance notice of safety-relevant changes and retirements, access to an
evaluation environment, and an achievable migration window.
Where version pinning exists, test and record the
pinned version. An endpoint name is not an identity.

Deployment checks and sentinel cases can catch some silent changes. They cannot prove that a silently updated
service is unchanged. When a provider will not expose a stable identity, say so in the submission and explain
how risk is still controlled. A provider's general benchmark report is not device evidence.

For consequential functions, predefine the response to an unexpected update: restrict the affected capability,
compare against retained baselines, run targeted then broader tests, inform deployers, and revert where
possible. Rollback itself needs verification, and old models disappear, so the contingency plan should include

DeNovo Innovations Group | 19 October 2026 | 8
PUBLIC COMMENT | FDA-2026-N-7874

continuing care without the original service. None of this transfers the manufacturer's responsibility to the
model provider. [1]

Question 26 – Agentic devices and the action boundary
Appendix A.1 already names tool use, human checkpoints, tool failures, and prompt injection. [1] My
recommendation is to make the action boundary an explicit test object.

A model may propose an action. It should not be the thing that grants itself permission to perform it. For
consequential actions, enforce the declared resource, patient, operation, and context restrictions outside the
generating model's control, and check current authorization at the moment of execution, including after
delegation, retries, and revocation. NIST's zero-trust architecture gives a clean separation between policy
decision and policy enforcement. [5] I apply that separation to clinical agents, and this is how I build my own
compliance architecture: the model's output is an instruction to a permission layer, never the permission itself.
In a disposable database harness for one administrative function of my own application, I tested that boundary
directly: authority is read from stored state when the operation executes, so an operation retried after the actor's
permission was revoked was refused, including a replay of one approved before the revocation, and instruction
text placed in the request's claims did not grant authority.

Deterministic enforcement handles the explicit constraints: allowed operations, scope, expiry, required
approvals. It does not establish clinical appropriateness, and it can fail through bad policy, implementation
defects, stale state, or bypass paths. Clinical evaluation, security testing, and human factors remain separate and
necessary.

Test the boundary through real enforcement paths. An authorized operation should succeed; the matching
unauthorized one should be blocked for the stated reason. A denial caused by a missing endpoint or a broken
fixture proves nothing. Verify the resulting state independently; a tool acknowledgment is not a postcondition.
[2] In a deliberately induced direct-database test of my own application's ledger, the caller received a result row
reporting a new balance, the connection was then closed before the transaction committed, and an independent
reader found the authoritative record unchanged. In the same tests, a fixture that silently failed to set the caller's
identity made an unauthorized action appear to succeed; a calibration step that printed the identity under test
caught it. Extend across multi-step trajectories: indirect prompt injection, manipulated tool outputs, unintended
delegation, duplicate actions, state changes between approval and execution, partial failure, attempted selfmodification of permissions, and safe interruption and recovery. Record every outcome. A blocked
unauthorized action is the enforcement layer working, and the attempt itself is a model failure worth
investigating. A necessary action wrongly denied is an enforcement failure. Both the attempt and the wrongful
denial belong in the record; only the block is a success.

Human checkpoints should count only when tested. FDA's May 2026 human factors guidance already sets a
risk-based frame for use-related evidence; the agent-specific extension is to challenge the review-andintervention workflow itself. [7] Run simulations with plausible but wrong recommendations, correct ones, and
appropriate deferrals, under representative workload and time pressure, and measure whether users catch
consequential errors and intervene in time.
Where meaningful review is infeasible, reduce the autonomy rather
than calling nominal review sufficient.

DeNovo Innovations Group | 19 October 2026 | 9
PUBLIC COMMENT | FDA-2026-N-7874

Skovos's comment addresses runtime permissions, institutional logs, and stop controls from the deployer's side.
[6] What I add is evidence that those controls discriminate between the right alternatives, operate on the
intended state, and fall back to something clinically safe. The concerns are complementary.

Implementation and proportionality
I propose no new document type. An evaluator-integrity record, an agent-action test supplement where relevant,
and a change-to-evidence impact assessment could live in existing quality-system records and test reports and
be supplied by reference.
Low-risk functions should not need the same external review, logging granularity, or
reassessment cadence as high-risk ones.

Retain enough to reconstruct the input, output, action, observation, and decision, with patient data minimized
and access controlled.
This does not require chain-of-thought disclosure. A checksum identifies retained bytes;
it does not establish completeness, custody, or source adequacy. [2] Responsibilities should connect
manufacturers and deployers without making institutions responsible for validating a supplier's model.

Limitations
These recommendations are engineering controls drawn from my own systems work, the FDA discussion
paper, and established security architecture practice. They are design proposals, not a clinically validated
package, and I do not claim a measured reduction in patient harm. The temporal-retrieval pattern was
implemented and exercised in a 2025 prototype; its date handling has not yet been tested or formally validated.
The authorization pattern was enforced and tested at the database layer of one application; delegation, patientand context-level restrictions, model-proposed actions, and use as a device control remain untested. The
human-oversight and supervisor-correlation recommendations are testable proposals rather than observed
performance. This comment was prepared with AI assistance; the interpretation and the decision to submit are
mine.

Future Direction
FDA could strengthen the evidence base by encouraging or supporting research on how often evaluators fail
silently in practice; how correlated device and supervisor errors are; how well human oversight performs under
realistic clinical workload; and what targeted independent review costs and what assurance it provides.

Conclusion
The competency-based approach is a real opportunity to connect clinical performance to evidence a reviewer
can inspect. I ask the Agency to make four things explicit as the work continues: that the evaluation itself is a
tested object, that evidence carries a date, that permission to act is enforced outside the model, and that human
oversight is demonstrated rather than presumed. The goal is not more paperwork. It is evidence that supports
the claim actually being made.

DeNovo Innovations Group | 19 October 2026 | 10
PUBLIC COMMENT | FDA-2026-N-7874

References
[1] U.S. Food and Drug Administration, Center for Devices and Radiological Health. Considerations for the Regulation of
Generative AI-Enabled Medical Devices: Discussion Paper and Request for Feedback. August 2026. https://
www.fda.gov/media/194242/download
[2] Maida G. Evaluator Integrity, Incomplete Evidence, and Traceable Assessment Conclusions: A Public Comment on NIST
AI 200-2 ipd. Version 1.0, October 5, 2026. Zenodo. https://doi.org/10.5281/zenodo.23167582 (record: https://
zenodo.org/records/23167582)
[3] National Institute of Standards and Technology. Building Evaluation Probes into Agentic AI. Project page, updated May 5,
2026. https://www.nist.gov/programs-projects/building-evaluation-probes-agentic-ai
[4] U.S. Food and Drug Administration. Marketing Submission Recommendations for a Predetermined Change Control Plan
for Artificial Intelligence-Enabled Device Software Functions. Final guidance, August 2025. https://www.fda.gov/
regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predeterminedchange-control-plan-artificial-intelligence (issued August 18, 2025; originally issued December 4, 2024, 89 FR 96259)
[5] Rose S, Borchert O, Mitchell S, Connelly S. Zero Trust Architecture. NIST SP 800-207. August 2020. https://doi.org/
10.6028/NIST.SP.800-207
[6] Skovos. Comment of Skovos on FDA-2026-N-7874: Considerations for the Regulation of Generative AI-Enabled Medical
Devices. September 29, 2026. Regulations.gov comment FDA-2026-N-7874-0130, attachment 1 (posted September 30,
2026). https://www.regulations.gov/comment/FDA-2026-N-7874-0130
[7] U.S. Food and Drug Administration. Content of Human Factors Information in Medical Device Marketing Submissions.
Final guidance, issued May 29, 2026 (91 FR 32061). https://www.fda.gov/regulatory-information/search-fda-guidancedocuments/content-human-factors-information-medical-device-marketing-submissions

DeNovo Innovations Group | 19 October 2026 | 11