OLIN (Xavier Molina)
Themes it raises
FDA questions it names
Q1 · The two-axis risk frameworkQ2 · The spectrum of device activityQ3 · When an output becomes directiveQ4 · Generalist and specialist usersQ5 · Multi-turn conversations that migrateQ6 · Care escalation functionsQ7 · The competency-based approachQ8 · Mapping the risk grid to evidenceQ9 · The benchmarking structureQ10 · Benchmark contamination and saturationQ11 · Clinical confirmation without a prospective trialQ12 · Statistically meaningful performanceQ13 · Synthetic dataQ14 · Comparators and acceptance criteriaQ15 · Performance against usual careQ16 · Independent third partiesQ17 · Devices with many functionsQ18 · Trading premarket certainty for postmarket monitoringQ19 · Postmarket performance evaluationQ20 · Machine-based supervisory agentsQ21 · Clinicians, institutions and societiesQ22 · Re-benchmarking after a modificationQ23 · PCCPs for GenAI devicesQ24 · Third-party foundation model changesQ25 · Foundation Model Master FilesQ26 · Agentic devices
The comment as filed
See attached file(s)
Attachment
CEO Comments to FDA on
Generative AI-Enabled
Medical Devices
October 18, 2026
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Submission
Submission
To: Dockets Management Staff, Food and Drug Administration —
Attention: Digital Health Center of Excellence, Center for Devices
and Radiological Health (CDRH)
Docket FDA-2026-N-7874
No.:
Re: Considerations for the Regulation of Generative AI-Enabled
Medical Devices: Discussion Paper and Request for Feedback
From: Xavier Molina, CEO & Co-Founder, OLIN
Date: October 18, 2026
Introduction and summary
For AI in medicine, OLIN follows one principle: trust, but verify.
The discussion paper moves in exactly that direction. OLIN supports it and
offers five recommendations to make it hold up on a real hospital floor.
I am the Co-Founder and Chief Executive Officer of OLIN, an independent AI
validation company for regulated industries. We donʼt build AI. We build the
evidence that shows whether a model works for the exact workflow, population
and regulatory setting where it will run.
I spent more than two decades in law: as general counsel of a technology
company, later as a fractional general counsel advising Silicon Valley tech
CEOs, as a litigation and commercial transactions attorney in the Bay Area, and
as an adjunct professor at UC Law San Francisco teaching advanced appellate
advocacy where law students wrote briefs on nuanced First Amendment issues
pending before the U.S. Supreme Court. Litigation teaches one lesson fast. A
claim is worth only the evidence that survives cross-examination. Clinical AI
deserves the same standard.
OLIN began in 2020 as a software quality assurance and test automation firm.
Today we apply that testing discipline, with machine learning evaluation and
human-factors methods, to AI-enabled medical software. From that seat, we see
the same pattern again and again.
Models that look brilliant on a benchmark break in the workflow. Models that
perform well still fail when an exhausted clinician at 3 a.m. misreads them,
over-trusts them or works around them.
Docket No. FDA-2026-N-7874 Page 1 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices General comments
Our five recommendations:
1. Validate the human–AI system, not just the model.
A GenAI device can ace every benchmark and still fail at the glass, where a
person reads the output and acts. Add a human-factor competency covering
automation bias, trust calibration, deskilling, indication creep and updaterelated mode errors.
2. Sharpen the two-axis risk framework.
Add human reconstructability, reversibility and behavioral volatility as
risk modifiers. Publish tiers that tie risk to evidence depth.
3. Anchor every benchmark to intended use.
Acceptance criteria should name the workflow, the user and the errors that
are never acceptable. Averages hide the patients who get hurt.
4. Recognize qualified independent validators.
Set clear independence and competence criteria and let them hold
sequestered benchmarks and run adjudication.
5. Make post market monitoring a condition of staying on the market.
Use the premarket benchmark as the baseline, with defined triggers,
override tracking and foundation model change detection.
General comments
A GenAI-enabled device needs evidence in four dimensions at once.
The paper covers trustworthy results and regulatory controls in depth. It covers
clinical fit in part and touches human factors only in passing.
OLIN validates AI-enabled medical software across these four dimensions.
A benchmark score is a hint, not a verdict.
A model is validated when the sponsor can show it improves a specific task, for
a specific user, in a specific clinical context, and that people can use it safely as
it changes.
Docket No. FDA-2026-N-7874 Page 2 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices General comments
Where the paper What we
Dimension Core question
addresses it recommend adding
Business and Does the device solve a Intended use; Workflow-anchored
clinical fit defined workflow comparators and acceptance criteria and
problem for a defined acceptance criteria predefined unacceptable
user? (Q14–15); clinical errors for each intended use
confirmation
Trustworthy Can users rely on Benchmarking elements Failure-mode profiling,
results outputs within defined S.1–S.3, E.1–E.4, R.1–R.2 repeated-run variance, and
limits? temporal and site validation
Regulatory Is the device governed Modifications, PCCP and Model lineage with signed
and quality by documented, Foundation Model artifacts, version pinning
compliance auditable controls? Master Files (Q22–25) and traceability from
benchmark result to release
Human-factors Can intended users In passing: E.4 A dedicated human-factor
safety in use understand, challenge (communication, competency and simulatedand safely act on outputs automation bias) and use testing in clinical
over time? human–AI team confirmation
comparators (Q14)
Human factor is the largest gap
GenAI makes human-factor risk larger, not smaller.
Fluent, confident language invites trust that the evidence may not support.
Long outputs add cognitive load under time pressure. Frequent model updates
change behavior that users have learned to expect. True safety is not locked
inside the algorithm. It is built on glass.
In our work we test for seven recurring human–AI risks:
• Misinterpretation and misperception: users miss or misread an output
because of layout, framing or unclear uncertainty.
• Trust miscalibration: confident wording or precise numbers create false
certainty, or undue doubt.
• Automation bias: users accept an output they should question.
• Deskilling: long-term reliance weakens the vigilance needed when the
device fails.
• Technostress and workflow friction: unclear handoffs or excess alerts add
burden instead of removing it.
• Indication creep: the device drifts into users, settings or decisions outside
its intended use.
Docket No. FDA-2026-N-7874 Page 3 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
• Change and mode errors: updates shift labels, thresholds or behavior, and
users act on outdated expectations.
Consider a triage tool cleared to flag possible cases for review. Within months,
overloaded clinicians start using it to rule cases out. Non-specialists apply it to
patients it was never tested on. The software has not changed. The human
behavior around it did. That is indication creep, and no benchmark will catch it.
Deskilling is quieter. If clinicians spend two years pressing “approve,” the
human backup is no longer a backup when the model drifts or goes offline. A
validation program must prove the algorithm works when the power is on, and
that the human stays sharp when it is off.
A device can pass every benchmark element and still fail on all seven. FDA
already has the tools to close this gap. Its guidance Applying Human Factors and
Usability Engineering to Medical Devices and IEC 62366-1 give an established
method. We recommend the GenAI framework reference them explicitly.
Evidence depth should scale with risk
We scale validation depth across three tiers, and the paperʼs two axes map onto
them cleanly.
Lower-risk functions need process reliability, clear user guidance and misuse
monitoring. Medium-risk functions with human override need robust
performance testing, subgroup analysis and trust-calibration testing. Higherrisk functions that shape diagnosis or treatment need rigorous clinical
validation, external and temporal validation, intensive monitoring and strict
update controls. The four dimensions stay the same at every tier. Only the depth
changes.
Responses to FDA’s questions
We answered all 26 questions. Numbering follows the discussion paper.
Section IV — Assessment of risk
Question 1: Does the two-axis framework capture the right dimensions?
Yes. Device activity and consequences are the right primary axes. We would add
modifiers rather than more axes:
• Human reconstructability under time pressure. Can the intended user
check the output against a source in the time actually available?
An output the user cannot verify behaves like an action-taking output,
whatever its label says.
Docket No. FDA-2026-N-7874 Page 4 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
• Reversibility and time to detection. An error caught in minutes differs
from one found after treatment starts.
• Behavioral volatility. How often does the deviceʼs behavior change
through updates, including upstream foundation model changes?
Frequent change raises the risk of change and mode errors.
• Verified downstream safeguards. A safeguard should count toward lower
risk only if it has been tested under realistic workload.
In our experience, downstream safeguards are the risk factor most often
assumed and least often tested.
Question 2: How should the continuum of directiveness be handled?
We agree directiveness is a continuum. Several output traits move a function
toward action-directing: ranking options, presenting one recommendation,
imperative or highly confident wording, false numerical precision, missing
citations, and pre-filled fields in the workflow.
Honesty should beat authority: a high-confidence output and a low-confidence
guess should never look identical on screen.
Weʼve observed that consumer-facing AI health recommendation product
positioned as informational, much of the test effort goes to keeping its
recommendations inside that boundary across complex user profiles. We
recommend FDA assess directiveness as the intended user experiences it,
through usability testing, not only as labeling describes it.
Question 3: Do patient-facing functions pose different or higher risk?
Yes, it is often higher, but not because patients are less capable. Patients usually
lack the context a clinician uses to catch an error, and they often act alone, with
no second reviewer. A fluent, confident answer carries extra authority for
someone who is anxious and has no way to check it.
Safeguards should match that reality without talking down to patients:
• Plain-language uncertainty, with a clear statement of what the device
cannot assess.
• Specific, actionable escalation: which symptoms mean “seek care now,”
and where to go.
• Testing across health literacy levels, languages and dialects (element R.2),
including simulated-use studies with real patients.
• Scope boundaries are enforced in the product itself, not only in the terms of
use.
Docket No. FDA-2026-N-7874 Page 5 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
Patients are capable.
They are also alone with the screen at 2 a.m.
The design should account for both.
Question 4: Should the generalist versus specialist distinction inform risk?
Yes. The same output carries different risks depending on who reads it. A
specialist can spot a plausible but wrong recommendation in their own field. A
generalist often cannot, so for that user the output behaves more like an actiontaking one. This is human reconstructability again.
Safeguards for generalist users:
• Flag plainly when an answer falls in a specialist area.
• Show sources and the key reasoning, so the user can check before acting.
• Prompt referral or specialist consultation for high-consequence decisions.
• Validate with generalist users, not only with specialists who already know
the answer.
This also guards against indication creep. A tool validated with specialists often
ends up in generalist hands.
Question 5: How should multi-turn conversations be handled?
Intended use should be defined by the most directive state a conversation can
reach, not by its opening state. The intended use statement should name that
boundary and the required behavior at it: refuse, redirect or escalate (element
S.2).
Benchmarks should include steering scenarios, where a user pushes the
conversation toward action-directing output over many turns. Test suites
should use medically complex user profiles to probe exactly these boundaries.
Single-turn tests rarely reveal these failures.
Question 6: How should under-escalation and over-escalation be weighed?
Measure them separately and never collapse them into a single accuracy score.
Under-escalation can cost a patientʼs life within hours.
Over-escalation causes harm more slowly: unnecessary visits, cost, anxiety, and
alert fatigue that teach clinicians to ignore the device.
We recommend:
• Separate acceptance criteria for each, set by intended use and clinical
setting.
Docket No. FDA-2026-N-7874 Page 6 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
• A hard ceiling on under-escalation for time-critical conditions, whatever
the over-escalation trade-off.
• Tracking over-escalation in post-market monitoring as a human-factors
signal. Rising override rates on escalation alerts are an early sign that users
are tuning the device out.
• Clinician and patient input when setting the trade-off, since acceptable
trade-offs differ across settings.
A device that cries wolf does not stay safe. Users stop listening.
Section V — Premarket evaluation
Question 7: Is benchmarking followed by clinical confirmation a useful framework?
Yes. Benchmarks are a modelʼs gym selfie. Clinical confirmation is the
marathon. The pairing mirrors how we structure validation engagements. We
suggest two refinements. Benchmarks should come from the intended use and
workflow, not from general clinical knowledge alone. Clinical confirmation
should test the human–AI system in that workflow, not the device in isolation.
Question 8: How might the two-axis risk framework help determine the level of evidence
needed?
Use the two axes to set evidence depth up front, before testing starts. The teams
that struggle at review usually scoped to the wrong tier. They treated a mediumrisk device like a lower-risk one because the bar was easier.
We suggest the axes decide three things: 1) which benchmarking elements
apply, 2) how strict the acceptance criteria are, and 3) how far up the clinical
confirmation ladder the sponsor must go. Here is a practical mapping, close to
the three-tier model we recommend:
Risk level Position on the two axes Benchmarking Clinical confirmation
Lower Non-directive output; low Core safety (S.1–S.3), clinical Retrospective evaluation plus
consequence if wrong knowledge (E.1) and usability checks
communication (E.4)
Medium Action-directing with All elements, including Shadow deployment plus
human override; subgroup analysis (R.2) and simulated-use studies with
moderate consequence a human-factors element representative users
Higher Action-taking, or high All elements, with stricter Multi-site clinician
consequence with low thresholds, sequestered test adjudication or a prospective
human reconstructability sets and independent study, with external and
adjudication temporal validation
Docket No. FDA-2026-N-7874 Page 7 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
Two refinements.
First, the modifiers in our Question 1 response should move a device within this
map. A high consequence output a clinician can verify in seconds needs less
evidence than one nobody can reconstruct.
Second, the tier should set thresholds for unacceptable errors, not only average
performance targets. At the higher tier, a single category of unacceptable error
should be disqualified, whatever the average.
Sponsors should state the tier and its justification at the start of the submission,
so reviewers can check the scoping before they read the results.
Question 9: Are benchmarking elements missing?
We recommend four additions:
1. A human-factor element (e.g., H.1 Human–AI interaction safety).
Does the output design support calibrated trust, keep uncertainty visible,
preserve user vigilance and allow safe override?
Test it with representative users, including scenarios where the device is
wrong, incomplete or inconsistent, and fallback tests where it suddenly
goes offline.
If clinicians can no longer perform without it, the device has become a
crutch.
2. Failure-mode characterization.
Pass rates hide how a device fails. Sponsors should profile failure severity,
clustering and plausibility.
A confident wrong answer that looks right is more dangerous than an
obvious error. Finding these latent failure modes is central to our
methodology.
3. Site and setting generalization.
Evidence should span distinct health systems, care settings and equipment.
A model tested in one hospital has proven only that it works in one room.
Where evidence comes from a single site, claims and labeling should be
limited to match.
4. Behavioral stability across versions.
The same benchmark should be rerun on each release, with differences
reported, not only absolute scores.
Useful standards to leverage include ISO 14971, AAMI CR34971 (applying ISO
14971 to machine learning), IEC 62366-1, IEC 62304 and ISO/IEC 42001.
Docket No. FDA-2026-N-7874 Page 8 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
Question 10: How can sponsors show benchmark performance predict real-world
performance?
Contamination and saturation are real risks with public benchmarks. Testing on
contaminated data is like letting a student write the exam after studying the
answer key. We recommend:
• A sequestered, independently held test set for each benchmark domain,
refreshed on a schedule.
• Temporal splits backed by dissimilarity checks, so the “future” test set is
not a light edit of the training data.
• A simple memorization baseline, such as a nearest-neighbor lookup. If it
scores almost as well as the device, the benchmark is measuring recall, not
competence.
• Reporting the gap between public and sequestered results as a
contamination signal.
• Repeated runs with rephrased inputs, reporting variance as well as means
(element R.1).
Sponsor-built benchmarks have a valuable role, because they reflect the actual
workflow. They should document item provenance and authorship and be
paired with an independent held-out subset.
Question 11: How should sponsors choose a clinical confirmation approach?
Choose it by risk tier, and design it to generate evidence, not momentum. Pilots
run by engaged clinical champions often show results the same tool cannot
repeat at scale, once the champion moves on.
Shadow deployment is underused and very informative for GenAI. It cannot
detect human-factor failures, though, because no one sees the outputs. For
medium- and higher-risk devices, we recommend pairing them with simulateduse studies under realistic conditions: night shifts, fatigue, interruptions and
competing alerts.
Retrospective evaluation alone is not enough for action-directing devices in
high-consequence settings.
Question 12: How can sponsors achieve statistically meaningful measurement?
Four practices help:
• Size test sets for rare, high-consequence errors, not the average.
Report confidence intervals for unacceptable-error rates, not only point
estimates. Zero errors in 200 cases still leaves a 95% upper bound of about
1.5%.
Docket No. FDA-2026-N-7874 Page 9 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
• Account for run-to-run variability.
GenAI outputs vary, so repeated runs per case belong in the design, with
variance reported (element R.1).
• Report synthetic and real data separately.
Never pool them into one score. Use synthetic data to find failure modes,
and real data to estimate how often they occur.
• Prespecify how the evidence combines.
The benchmark is the gate. Clinical confirmation shows performance in the
workflow. Strong results in one should not offset failures in the other.
The analysis plan should be fixed before testing starts. A benchmark analyzed
after the results are in is a story, not evidence.
Question 13: Where is synthetic data suitable?
Synthetic data works well for stress testing: rare conditions, adversarial inputs,
edge cases and steering scenarios that real data seldom contains.
It should not be the sole evidence of performance on real populations. Every
synthetic set should be checked against real data, so it does not reproduce the
gaps it was meant to fill.
Question 14: How should comparators and acceptance criteria be set?
The comparator should match the intended workflow. For assistive devices,
compare the human–AI team with the standard of care. Compare the device
alone only for functions that act alone.
The comparator clinician should match the intended user: a device meant for
generalists should be compared with generalists.
Acceptance criteria should have two parts: performance at least as good as the
comparator on average, and no increase in the errors defined in advance as
unacceptable. Average performance can improve while the most dangerous
errors become more frequent.
Results should also be reported by subgroup and by site. An average alone is
like saying the room is at a comfortable temperature while half the patients are
freezing and half are on fire.
Question 15: Should counterfactual comparators be used?
Yes, especially where the deviceʼs value is access, such as replacing a delayed
specialist review. The counterfactual must be realistic and include the delay,
and the error rate of the care patients would actually receive.
Docket No. FDA-2026-N-7874 Page 10 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
Question 16: Should independent third parties participate?
Yes. Independent validators can maintain sequestered benchmarks, run expert
adjudication, conduct conformity assessment and develop qualified test tools,
including human-factors protocols through the MDDT program. OLIN would
welcome a role in this ecosystem.
We suggest these independence and competence criteria:
• No role in developing the device being assessed, and no fees contingent on
the outcome.
• Demonstrated competence across clinical, machine learning evaluation,
regulatory and human-factors disciplines.
• An operating quality system, such as ISO 13485 or ISO/IEC 17025 for testing.
• Public disclosure of methods and conflicts of interest.
To protect competition: accredit multiple providers, publish the criteria, avoid
exclusive designations, keep sponsor choice and scale costs for small
companies.
Third-party review should add confidence, not create a bottleneck.
Question 17: Could the approach apply to multimodal models and world models?
Yes. The competency structure carries over, but three areas need more
attention:
• Acquisition variation.
Image-based devices fail on different scanners, lighting and protocols.
Benchmarks should cover the equipment and conditions of real sites, not
only the cleanest images.
• Cross-modal consistency.
A vision-language model can describe an image fluently while misreading
it. Tests should check that the text matches what is actually in the image
and catch shortcuts such as reading scanner labels or institutional artifacts
instead of anatomy.
• Prediction over time.
For world models that forecast patient states or simulate outcomes,
sponsors should validate predictions against observed outcomes over time,
not only against expert opinion.
Multimodal output also raises the human-factorsʼ stakes. A confident caption
next to an image lends it authority the image itself may not support.
Docket No. FDA-2026-N-7874 Page 11 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
Section VI — Postmarket monitoring
Question 18: When could postmarket monitoring offset premarket uncertainty?
Only when all four conditions hold:
• the consequence of error is low or moderate;
• errors are verifiable and reversible;
• monitoring has predefined triggers and a tested rollback; and
• a sample of real outputs gets regular human review.
Action-taking devices in high-consequence settings are not suitable.
Neither are patient-facing devices that inform high-consequence decisions without a
clinician in between. Where FDA does accept more premarket uncertainty, active
monitoring should be a condition of the authorization. If the monitoring stops working,
the authorization should pause.
Question 19: How should postmarket approaches be assessed, and with what cadence and
triggers?
The three approaches complement each other, and higher-risk devices need all three.
We recommend a fourth source of evidence: human-behavior signals. These include
override rates, acceptance without edits (a proxy for automation bias), escalation
patterns, fallback-path use and confusion reports after updates.
Model metrics show whether the device is drifting. Behavior signals show whether its
users are. High override rates are not a human problem. They are a signal that the
model is drifting or failing in practice. Decision-time telemetry matters too: if users
read a summary and close the chart in four seconds, a triage tool is being used as a
diagnosis engine.
Cadence should follow the risk tier. Re-benchmarking should run at every model
change, at every foundation model version change and on a fixed schedule. Triggers
should include drift beyond a set threshold, a significant change in override rates, a
new site or population, and any safety incident.
Question 20: Can supervisory agents support monitoring?
Yes, for scale. A supervisory agent is itself a measurement instrument and should be
validated as one. It needs its own benchmark, measured agreement with clinician
adjudicators and periodic human audit. It should not share a foundation model with
the device it monitors. A supervisor built on the same model shares the same blind
spots.
Question 21: What roles should the wider ecosystem play?
Health systems are best placed for local validation and site-specific feedback. Specialty
societies can contribute benchmark content. Independent validators can provide
continuity across sites and versions. Manufacturers stay accountable. Their contracts
with deploying sites should define the data sharing that monitoring requires.
Docket No. FDA-2026-N-7874 Page 12 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Responses to FDA’s questions
Questions 22 and 23: How should modifications and PCCPs work for GenAI?
Re-benchmarking should be scaled with impact. A change to the foundation model, the
intended use or the way outputs are presented should trigger full re-benchmarking.
Changes to prompts or retrieval sources can use a targeted subset.
Changes to how outputs are presented are human factors change. They should trigger
usability re-evaluation, because they are a common source of change and mode errors.
When future changes cannot be fully prespecified, we recommend a simple principle:
specify the test, not the change. A GenAI PCCP could fix the performance envelope,
the benchmark battery, the acceptance criteria and the rollback procedure. Any change
that passes inside that envelope is managed under the plan. Any change that falls
outside it goes to FDA. Done well, a PCCP is not red tape. It is a shared contract that lets
good models improve quickly and safely.
Question 24: How can manufacturers handle foundation model changes they do not control?
Three layers work together:
• Contracts: version pinning, advance notice of changes, deprecation windows and
access to change logs.
• Technical controls: a fixed canary test suite run on a schedule to detect silent
behavior changes, plus structured logging of model versions per output.
• PCCP: a prespecified re-benchmark that runs automatically on any upstream
version change.
Silent upstream changes are among the hardest failures to see without a fixed canary
suite. We recommend FDA treat that suite as a baseline expectation.
Section VII — Other topics
Question 25: Would Foundation Model Master Files be useful?
Yes, and they would help smaller sponsors most. In addition to the contents of
FDA lists, we suggest each file include versioned behavioral results on a stable
test set, a deprecation schedule, and change notification commitments with
timelines.
Question 26: What additional considerations apply to agentic devices?
Agentic devices need four additional controls:
• human confirmation gates before high-consequence actions;
• complete, auditable logs of every tool call;
• acceptance criteria per step and end to end, since small errors compound
across steps; and
• red-team testing for prompt injection.
Supervising an agent is also a human-factors task.
Docket No. FDA-2026-N-7874 Page 13 of 14
OLIN — CEO Comments to FDA on Generative AI-Enabled Medical Devices Proposed next steps for FDA
Vigilance declines when a system is usually right, and handoff errors rise.
Clinical confirmation should test that supervision directly.
Proposed next steps for FDA
Six actions would turn the discussion paper into a workable framework.
We donʼt need more bureaucracy. We need smarter boundaries.
1. Add a human-factors competency domain to any draft guidance, crossreferenced to FDAʼs human factors guidance and IEC 62366-1.
2. Publish worked examples that place common GenAI functions on the twoaxis framework, with the evidence expected at each risk level.
3. Limit claims to the evidence. Where validation comes from a single site or
skips subgroup analysis, narrow the label accordingly.
4. Pilot a third-party benchmarking and adjudication program through the
ASCA or MDDT pathways, open to multiple qualified providers.
5. Make active post market monitoring a condition of continued
authorization for higher-risk GenAI devices, with override tracking and a
foundation model canary suite as baseline expectations.
6. Hold a public workshop on evaluating human–AI teams, bringing
together sponsors, health systems and independent validators.
Closing
OLIN is ready to help.
We can share our human-factor test protocols, our benchmark design approach
and our monitoring signal definitions with FDA staff. We would welcome a role
in a third-party pilot or a technical briefing. GenAI will reach patients fast.
The evidence that it is safe in human hands must arrive at the same speed.
Trust, but verify.
Thank you for the opportunity to comment.
Xavier Molina
CEO & Co-Founder, OLIN
xavier@olinadt.com ·
(408) 565-8063
Docket No. FDA-2026-N-7874 Page 14 of 14