Tanmaya Kumar (Behavioral Health Open Source)

This comment responds to discussion questions 1, 19, 20, 21, and 26. A fuller comment is attached.

INTEREST AND DISCLOSURE

I maintain an open source healthcare audit standard and its reference implementations, published under Apache 2.0 at github.com/bh-healthcare. The work concerns how clinical systems record actions taken by AI agents, and how such records name the human accountable for an agent’s action. I am not a device manufacturer and have no product before the Agency. I have no commercial interest in this proceeding, and the artifacts referenced are published openly at no cost. I am an individual member of HL7 International with an open technical contribution before the HL7 Security and Privacy Work Group on the matter described below.

SUMMARY

The activity axis in Figure 1 distinguishes an action-taking function operating under healthcare professional supervision from one operating fully autonomously. That distinction is central to the proposed risk framework. It is also, at present, not verifiable from the records a clinical system produces.

Healthcare audit standards, including HL7 FHIR R5 AuditEvent, model a single actor per event. That model was correct for as long as the identity that authenticated, the entity that acted, and the person whose intent was carried out were the same human. An AI agent separates those three, and a single actor field collapses them. A device authorized on the basis that it acts under continuous professional supervision, which then operates without it, produces records indistinguishable from the supervised case. The drift is not merely difficult to detect. It is not represented in the record at all.

QUESTION 1: TRACEABILITY OF ATTRIBUTION AS A FRAMEWORK DIMENSION

Traceability of an output to its source materials and traceability of an action to an accountable person are distinct concerns. The second is load-bearing for the activity axis, because a function’s position on that axis is a claim about whose clinical intent an action represents. Three identities separate when an agent acts: the authenticating identity, whose credentials were presented; the acting identity, which entity performed the action; and the authorizing identity, whose intent the action represents. A framework dimension for attribution traceability would ask whether a device’s records preserve that distinction, and would place a device that cannot at higher risk than one that can, holding activity and consequence constant.

The attached comment discusses a public reference implementation that treats traceability thoroughly and still cannot produce a record from which the supervision relationship is recoverable, because its trace lives in platform telemetry rather than in the audit record.

QUESTION 20: ATTRIBUTION AS A PRECONDITION FOR SUPERVISORY AGENTS

A supervisory agent monitoring a subject agent is one automated system acting with respect to another. HL7 publishes an extension, auditevent-OnBehalfOf, whose description names the agent-acting-for-agent case, but whose value binding permits only Organization, Patient, Practitioner, PractitionerRole, RelatedPerson, and CareTeam. Device is excluded, while AuditEvent.agent.who permits Device, so the case the description names is not expressible. The extension carries no child elements, so the requestor flag remains on the agent, and a requestor=true access review returns the machine rather than the accountable clinician. It is marked Draft, maturity level 1, and used by no profile in the specification. I raise this to identify a specific and remediable gap, not as a criticism of HL7’s process.

QUESTIONS 26 AND 19: TESTABLE ACCEPTANCE CRITERIA

Appendix A element A.1 names compliance with human-oversight checkpoints before irreversible or high-consequence actions. I suggest that compliance also needs to be recorded in a form that survives to postmarket review, and that three properties are testable candidates: that attribution is emitted rather than inferred by a later reviewer; that an agent action without a resolvable authorizing human is positively representable and distinguishable from an omitted field; and that the strength of an attribution is stated, so that an identity derived from an issued token does not serialize identically to one asserted by the agent itself. These are properties of the emitted record rather than of the model, so they can be evaluated without access to model internals or to a third-party foundation model’s architecture.

QUESTION 21: STANDARDS BODIES

The gap above has an identified owner and an open technical process. Agency interest expressed as an expectation about what records should preserve would likely move it faster than new regulatory requirements would.

AVAILABILITY

The specification, the FHIR R5 profile, and the reference implementations are at github.com/bh-healthcare, with a technical report at doi 10.5281/zenodo.21682867.

Tanmaya Kumar

Comment on Docket No. FDA-2026-N-7874
Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and Request for Feedback
Submitted by: Tanmaya Kumar, Behavioral Health Open Source (bh-healthcare.org)Date: August 26, 2026Responding to discussion questions: 1, 19, 20, 21, 26

Interest and disclosure
I maintain an open source healthcare audit standard and its reference implementations, published under the Apache 2.0 license at github.com/bh-healthcare. The work addresses how clinical systems record actions taken by AI agents, specifically how such records name the human accountable for an agent’s action.
I am not a device manufacturer and have no product before the Agency. I have no commercial interest in the outcome of this proceeding. The artifacts referenced below are published openly, at no cost, with no associated commercial offering. I am an individual member of HL7 International and have an open technical contribution before the HL7 Security and Privacy Work Group concerning the matter described in Section 3.
This comment concerns one gap that runs through several of the Agency’s discussion questions, and it is offered because the gap is mechanical and checkable rather than a matter of opinion.

1. Summary
The activity axis in Figure 1 distinguishes an action-taking function operating under healthcare professional supervision from one operating fully autonomously. That distinction is central to the proposed risk framework. It is also, at present, not verifiable from the records a clinical system produces.
Healthcare audit standards, including HL7 FHIR R5 AuditEvent, model a single actor per event. That model was correct for as long as the identity that authenticated, the entity that acted, and the person whose intent was carried out were the same human. An AI agent operating clinical software separates those three, and a single actor field collapses them. The record that results cannot distinguish an action a clinician authorized from one the agent took on its own.
The consequence for this framework is direct. A device authorized on the basis that it acts under continuous professional supervision, which then operates without that supervision in practice, produces audit records that are indistinguishable from the supervised case. The drift is not merely hard to detect. It is not represented in the record at all.
I suggest that traceability of attribution belongs in the risk framework as the Agency invites in Question 1, that it is a precondition rather than a feature for the supervisory-agent approach contemplated in Question 20, and that it is a specific and testable acceptance criterion for the agentic devices addressed in Question 26.

2. Response to Question 1: attribution traceability as a framework dimension
Question 1 asks whether additional dimensions, including the traceability of the output, should be represented in the risk framework.
I suggest that traceability of the output to source materials and traceability of the action to an accountable person are distinct concerns, and that the second is load-bearing for the activity axis specifically.
The activity axis positions a function according to how independently it directs or takes action. The paper states that a function acting with continuous professional supervision is meaningfully different from one acting in fully autonomous fashion. That is correct and it is the right distinction to draw. It also creates an evidentiary obligation: for the distinction to carry regulatory weight across the total product life cycle, a deployed system must be able to demonstrate which side of it a given action fell on.
Three identities separate when an agent acts in a clinical system, and each answers a different question:
Authenticating identity. Whose credentials were presented to the target system.
Acting identity. Which entity performed the action.
Authorizing identity. Whose clinical intent the action represents.
For direct human action all three coincide, which is why one field sufficed. For an agent operating under a clinician’s authorization they are three different things, and the position of the function on the activity axis is a claim about the third one.
A framework dimension for attribution traceability would ask whether a device’s emitted records preserve the distinction between these identities, and would place a device that cannot preserve it at higher risk than one that can, holding activity and consequence constant. The rationale is that supervision claimed at authorization but unverifiable in deployment is not equivalent to supervision that can be shown.
A concrete illustration. A publicly documented reference implementation, FHIR Agent Studio, demonstrates the gap without any of the shortcuts that would make it an unfair example. It comprises twelve clinical agents running against a synthetic FHIR repository on a widely deployed health data platform, and its author describes it as a prototype rather than a product. Its treatment of transparency is thorough. Each run retains the assembled prompt and the raw model response, a replayable trace of the full message flow including the record read, the query, the retrieval step, and the model call, and a response history recording each model output with its prompt, a timestamp, and a hash. Every action the agents propose is a draft that a clinician approves before anything is written.
That design is careful, and the constraint it runs into is structural rather than a defect of the implementation. The trace is held in the platform’s interoperability layer, not in the standard audit record. It does not travel with the clinical data and it is not what an oversight query reads. At the approval step, what enters the clinical record is a resource attributed to the approving clinician. The agent’s participation persists only in vendor-specific platform telemetry, correlated to the clinical record by nothing durable within it. Reading the record afterward, an investigator cannot distinguish a resource the clinician composed from one an agent drafted and the clinician approved.
This is precisely the human-supervised band of the activity axis in Figure 1. A well-built implementation, on a mainstream platform, operating exactly as intended, produces a record from which the supervision relationship cannot be recovered. The retention of prompts and model outputs illustrates a related point: that content is appropriate for a synthetic corpus, but persisting it over real patients creates a disclosure surface subject to minimum-necessary and, for substance use disorder records, 42 CFR Part 2 constraints. Attribution is better carried as scalar metadata in the audit record than as retained model content, because the metadata form can be preserved indefinitely without creating that surface.

3. Response to Question 20: attribution is a precondition for machine-based supervisory agents
Question 20 asks whether postmarket monitoring can be facilitated by machine-based supervisory agents, and what considerations, including the evaluation and reliability of the supervisory agent itself, the Agency should take into account.
A supervisory agent monitoring a subject agent is an instance of one automated system acting with respect to another. If the supervisory agent takes any action, whether flagging, escalating, or intervening, its own actions enter the record. The record must then represent an agent acting in a chain, under authority that traces back to an accountable human somewhere.
The current standard cannot express this. HL7 publishes an extension, auditevent-OnBehalfOf (canonical http://hl7.org/fhir/StructureDefinition/auditevent-OnBehalfOf, version 5.3.0, standards status Draft, maturity level 1). Its description reads: “When an AuditEvent is attributed to an agent that is acting on behalf of another agent. Typically needed when multiple agents are acting on behalf of different organizations, and when PractitionerRole is not appropriate.”
The first sentence names the agent-acting-for-agent case. The second scopes the typical need to organizational affiliation. The artifact implements the second. Extension.value[x] is a required Reference bound to Organization, Patient, Practitioner, PractitionerRole, RelatedPerson, and CareTeam. AuditEvent.agent.who permits Device. A Device can therefore act, but cannot be named as the entity acted for, and the case the first sentence names is not expressible.
A second effect compounds this. The requestor flag is a sibling of who on the agent element. Extension.extension on this extension has a maximum cardinality of zero, so the extension carries no child elements and there is no location on the referent that could hold a requestor indication. Applying the extension to an agent action therefore leaves requestor on the agent. The most standard access-review query in FHIR, filtering on requestor = true, returns the machine rather than the accountable clinician. That query is the operation an investigator, an auditor, or a compliance officer performs.
The specification page states that the committee is seeking implementation feedback, and records that the extension is not used by any profile in the specification. The underlying change request, FHIR-40293, records the motivating use case as two practitioners working on behalf of different organizations, resolved as a non-substantive change in February 2023. An alternative considered in that ticket, an agent.agent element relating one agent to another, was set aside because no case before the work group at the time required it.
I raise this not as a criticism of HL7’s process, which behaved correctly given the cases before it, but to identify a specific and remediable gap between what the Agency is contemplating and what the interoperability standards currently support. A supervisory-agent monitoring architecture would produce records that cannot distinguish the supervisor from the supervised, and cannot trace either to the human whose authority they operate under.
Suggested consideration. Before a supervisory-agent approach is relied upon for postmarket monitoring, the Agency may wish to consider whether the records such an architecture produces are capable of expressing the supervisory relationship at all, and to encourage or reference standards work that closes the gap.

4. Response to Question 26: attribution as an acceptance criterion for agentic devices
Question 26 asks what additional considerations inform the evaluation of agentic GenAI-enabled devices, and how the elevated risk associated with autonomous multi-step action, tool use, and reduced opportunity for human review should be reflected in acceptance criteria and oversight.
Appendix A element A.1 already names the relevant behavior: compliance with human-oversight checkpoints before irreversible or high-consequence actions. That is the right thing to benchmark. I suggest that it also needs to be recorded, and recorded in a form that survives to postmarket review.
A benchmark demonstrates that the device honors an oversight checkpoint under test conditions. A record demonstrates that it did so on a particular occasion in production. The second is what postmarket monitoring reads, what an adverse event investigation reconstructs, and what distinguishes a device operating within its authorized envelope from one that has drifted outside it.
Three properties are testable and I suggest they are candidates for acceptance criteria for agentic devices:
Attribution is emitted, not inferred. The record states which entity acted and which human authorized it, rather than leaving the relationship to be reconstructed by a reviewer from timing or context. Reconstruction after the fact is not reliable and does not survive personnel change.
An unattributed agent action is representable and distinguishable. A device that acts without a resolvable authorizing human should produce a record that says so, positively, rather than a record that omits the field or a record that is not produced at all. An action that leaves no record is indistinguishable from an action that never occurred, and the denial rate is itself the clearest signal that an oversight mechanism is failing.
The strength of the attribution is stated. An authorizing identity derived from a token issued by an identity provider is a different evidentiary object from one asserted by the agent in the call. Both may be acceptable depending on risk profile, but they should not serialize identically, because a consumer cannot then tell a deployment running full identity verification from one accepting the agent’s own claim.
These are properties of the emitted record rather than of the model, which means they can be evaluated without access to model internals or to a third-party foundation model’s architecture. That may be useful given the visibility constraints the paper describes in Section II and Section VII.A.
Note also that footnote 24 already contemplates audit log availability as content for a voluntary Foundation Model Master File. The consideration above is the complement at the device layer: not whether logs exist, but whether their structure preserves the accountability relationship.

5. Response to Question 21: the role of standards-setting bodies
Question 21 asks what roles standards-setting bodies and other stakeholders might play in postmarket monitoring, and how those roles can be structured without diffusing manufacturer accountability.
The properties in Section 4 are semantic properties of an interoperability format. They cannot be established by any single manufacturer, because audit records cross organizational boundaries. A health information exchange, an electronic health record vendor, a payer, and a federal investigator all read records produced by systems they do not control. A manufacturer-specific attribution scheme reproduces the current problem in a proprietary shape and adds a translation burden.
This argues for the standards-setting body role being specifically the definition of attribution semantics, with manufacturer accountability preserved intact. The manufacturer remains responsible for what its device does and for what its device emits. The standard determines only whether the emitted record is capable of expressing what happened.
I would note one practical asymmetry. Small and under-resourced provider organizations, including community mental health centers, federally qualified health centers, and behavioral health programs, carry the same obligations as large health systems but cannot build audit infrastructure independently. If attribution semantics are settled in an open standard, those organizations inherit the capability. If they are settled per vendor, those organizations receive whatever their vendor supplies and are least able to evaluate it.

6. Response to Question 19: what postmarket degradation monitoring requires
Question 19 asks about approaches to postmarket performance evaluation, including performance degradation monitoring.
Section VI.C distinguishes intentional sponsor-initiated modifications, model-evolution changes that occur passively and incrementally, and unplanned changes arising from third-party foundation model updates. For an agentic device, there is a fourth category worth naming: change in the degree of human involvement in practice, without any change to the software at all.
A device may be deployed with an oversight checkpoint that clinicians initially exercise deliberately and, over months, come to acknowledge reflexively or bypass through workflow adaptation. Nothing in the device changed. Its position on the activity axis did. This is a well-documented pattern in clinical software, and the paper’s own discussion of automation bias in element E.4 acknowledges the underlying mechanism.
Degradation monitoring that reads audit records can detect this, but only if the records distinguish an action a human authorized from one the agent took alone. If they do not, the drift is invisible to precisely the monitoring the framework relies on.

7. Limits of what I am claiming
I want to be explicit about what this comment does not assert.
Attribution is not detection. A schema cannot detect a non-cooperating agent. An agent driving a user interface under a clinician’s credentials is indistinguishable, to the target system, from that clinician. Attribution must be emitted by the agent layer, which means it works where the deployment path is controlled and does not work where it is not. I have stated this limit in the published specification and state it here.
This is not a proposal for a new regulatory requirement. The properties in Section 4 are offered as candidates for consideration within the acceptance criteria the Agency is already contemplating for agentic devices, not as an argument for additional regulatory burden. They are, in my assessment, among the less burdensome available, because they concern the format of a record a compliant system already produces.
My own artifacts are not devices. The audit standard and its implementations are infrastructure for recording actions, not software functions that meet the device definition. I reference them for concreteness, not to place them before the Agency.

8. Availability
The specification, the FHIR R5 AuditEvent profile, the gap analysis referenced in Section 3, and the reference implementations are published under Apache 2.0 at github.com/bh-healthcare/bh-audit-schema and github.com/bh-healthcare/bh-mcp-attribution. The technical report describing the attribution model is deposited with a persistent identifier at 10.5281/zenodo.21682867.
I mention this only because the properties described in Section 4 are easier to evaluate against a concrete implementation than in the abstract. There is no commercial offering associated with any of it and I am not seeking one.
The implementation discussed in Section 2 is FHIR Agent Studio by Sean Connelly, documented at community.intersystems.com/post/introducing-fhir-agent-studio-ai-agents-fhir-intersystems-iris with source at github.com/SeanConnelly/ai-studio-for-fhir. I have no affiliation with that project or with its author, and cite it because it is public, well documented, and unusually thorough in its treatment of traceability. The observation in Section 2 concerns what the available standards allow such a system to record, not the quality of the work.
The specification of the extension discussed in Section 3 is published at hl7.org/fhir/extensions/StructureDefinition-auditevent-OnBehalfOf.html.
I appreciate the Agency’s decision to seek early input on this topic, and I am available to provide further detail on any point above.
Tanmaya Kumar Behavioral Health Open Source bh-healthcare.org