Xillium (Don Wickelgren)

Xillium staffs and supervises the human oversight layer around AI-driven administrative and clinical support workflows in provider organizations. We do not build devices and take no position on classification or market pathway. Our comment addresses one point running through the two-axis risk framework and the postmarket monitoring section: the framework treats human oversight as a property of the device, and in deployment it is a property of the deploying organization’s staffing. The attached comment describes how effective autonomy is set at deployment rather than at authorization, how oversight degrades under caseload without producing an error signal, and why escalation authority carries more safety value than review volume. We recommend that postmarket monitoring capture oversight as a stated condition of use through four deployment fields rather than as an assumption.

COMMENT ON CONSIDERATIONS FOR THE REGULATION OF
GENERATIVE AI-ENABLED MEDICAL DEVICES

Docket: FDA-2026-N-7874
Submitted by: Don Wickelgren, Founder, Xillium
Date: October 5, 2026
Discussion questions addressed: 1, 6, 19, 21, with reference to benchmarking element E.4

INTEREST OF THE COMMENTER
Xillium staffs and supervises the human layer around AI-driven administrative and clinical support
workflows in provider organizations, including health systems, MSOs, ACOs, and physician
groups. Our teams work inside client systems on prior authorization, denials, referral coordination,
patient outreach, care gap closure, and prescription refill support. We do not build devices, do not
develop clinical decision support, and take no position on the classification or market pathway for
any product.

We comment on one narrow point that runs through the two-axis risk framework in Section IV and
the postmarket monitoring approaches in Section VI: the paper treats human oversight as a
property of the device, and in practice it is a property of the deploying organization’s staffing. We
respond principally to Question 21, which asks what roles healthcare institutions and other
stakeholders might play in postmarket monitoring, and to Questions 1, 6, and 19.

1. EFFECTIVE AUTONOMY IS SET AT DEPLOYMENT, NOT AT AUTHORIZATION
(Question 1)
Question 1 asks whether the two-axis framework captures the dimensions most relevant to risk,
and invites additional dimensions including the availability of downstream safeguards and the time
pressure of the deployment setting. We suggest these two are not separate dimensions but
expressions of a single one: how the oversight step is staffed.

Section IV distinguishes a function that acts with continuous healthcare professional supervision
from one that acts in fully autonomous fashion. In our experience that line moves after
authorization. The same tool, deployed under the same labeled oversight requirement, operates at
materially different effective autonomy levels depending on who staffs the review and what that
person is permitted to do. Where the reviewer is trained, has time, and has standing authority to
stop the workflow, the device is supervised. Where the review step exists on paper and is absorbed
into an already-full queue, the device is effectively autonomous with a signature attached.

A framework that grades autonomy without accounting for this grades the design rather than the
deployment. We suggest the staffing of the oversight step be represented explicitly, rather than left
implicit in the activity axis.

Xillium comment, Docket FDA-2026-N-7874 Page 1
 2. OVERSIGHT DEGRADES UNDER LOAD BEFORE IT FAILS VISIBLY
(Questions 1 and 19; benchmarking element E.4)
Benchmarking element E.4 in Appendix A identifies automation bias, in which a user accepts an
output without appropriate scrutiny because of the device’s fluency or perceived authority. We
agree this belongs in the framework and suggest one extension: in our experience the dominant
driver is caseload, not fluency.

The failure mode we see is not a reviewer overriding the model incorrectly. It is a reviewer who
stops reading. This happens gradually, correlates with review volume rather than with any device
characteristic, and produces no error signal of its own. Approval rates stay stable. Throughput
improves. The oversight step continues to be documented.

Prior authorization and refill queues are where this is most visible to us, because volume is high,
each item looks routine, and the cost of pausing one falls on the reviewer. We would expect the
same dynamic in any deployment where a human confirmation step sits in front of a high-volume
output stream.

This has a consequence for the premarket and postmarket division of labor contemplated in Section
VI. Automation bias driven by caseload cannot be detected through benchmarking, because the
condition that produces it does not exist in the test environment. Postmarket monitoring built on
device output performance alone will not detect it either. The device is performing as authorized.
The oversight is not.

3. ESCALATION AUTHORITY MATTERS MORE THAN REVIEW VOLUME
(Questions 6 and 21)
Requiring human review of every output is not the right default and can reduce the benefit of tools
that already perform well. Universal review is a poor proxy for safety.

The variable that carries the safety value in our experience is not whether a human reads each
output. It is whether the human in the workflow can stop the process and reach clinical authority
without penalty, and whether they have been trained to recognize when to do so. Our standing
instruction to delivery staff is to pause and bring the question to the clinical authority. Recognition
and escalation, not independent judgment.

This bears on Question 6. Both directions of escalation error are shaped by who is positioned to
escalate. Under-escalation in the deployments we see is rarely a judgment failure by someone who
recognized the problem. It is the absence of anyone positioned and authorized to act on what was
recognized. Over-escalation, conversely, rises when the escalation path is unclear and staff route
defensively.

The distinction is actionable for the framework. A device deployed with a low-volume,
high-authority human checkpoint may warrant different postmarket expectations than the same
device deployed with a high-volume, no-authority one, even though both would be described as
human-in-the-loop.

Xillium comment, Docket FDA-2026-N-7874 Page 2
 4. SOME SAFETY-RELEVANT SIGNALS NEVER REACH THE DEVICE
(Question 1)
Question 1 raises downstream safeguards as a candidate dimension. We would add upstream ones.

In refill workflows specifically, the information that should stop a refill often arrives through a
channel the device cannot see. A patient mentions a new symptom during an outreach call. A
caregiver phones in a change. Staff notice something during scheduling that does not match the
chart.

Our teams sit on those channels. The signal reaches the clinical decision only if a person on that
channel is listening for it and is authorized to route it.

The regulatory implication is that oversight adequacy is partly a question of what non-device
channels are staffed around the device, not only what the reviewer does with the device’s output.

5. RECOMMENDATION
(Questions 19 and 21)
Question 19 asks what additional postmarket approaches CDRH should consider. We suggest the
postmarket framework capture oversight as a stated condition of use rather than as an assumption,
using a small, fixed set of deployment characteristics:

• Who performs the oversight step, by role and training level
• Review caseload per reviewer over the relevant period
• Escalation path, specifically whether the reviewer can halt the workflow and how they reach
clinical authority
• Adjacent channels staffed around the workflow, where safety-relevant information may arrive
outside the device

These are four fields, not a reporting program. The point of collecting them is narrow: when a
device is authorized on the assumption that a human is watching, the record should say what
“watching” was assumed to mean, so that a deployment operating outside those conditions can be
recognized as such.

On the accountability constraint in Question 21, we want to be clear about what we are not
proposing. These four fields are conditions of use, in the same sense as any other labeled condition
under which a device was found safe and effective. They keep accountability with the
manufacturer, which characterizes the oversight its device requires and remains responsible for
whether the device performs under those conditions. We are not proposing that FDA regulate
provider staffing, nor that deploying institutions assume a share of the manufacturer’s obligation.
Stating a condition of use does not diffuse accountability. Leaving it unstated is what makes
accountability difficult to locate when a deployment departs from what was assumed.

Xillium comment, Docket FDA-2026-N-7874 Page 3
 CLOSING
Our interest here is not in the market pathway for any device. It is that the oversight the framework
relies on is real work, performed by staff, under budget and volume pressure that the authorization
process does not see. We would welcome the opportunity to provide further detail on what
oversight looks like in deployment.

Don Wickelgren
Founder, Xillium
policy@xillium.com

Xillium comment, Docket FDA-2026-N-7874 Page 4