Arthrex Inc.

Attached comments are provided on behalf of Arthrex, Inc.

1
3
5
7
9
11
13
15
17
19
21
23
25
FDA Generative AI Discussion Paper Questions
Question
Section IV: Considerations for the Assessment of Risk for

Does the two-axis risk framework, organized around device activity and the consequence of
relying on an incorrect output, appropriately capture the dimensions most relevant to the
risk of a GenAI-enabled software function? If there are additional dimensions—such as the
reversibility of a resulting action, the availability of downstream safeguards, the time
pressure of the deployment setting, or the traceability of the output (i.e., to primary
source materials)—that should be represented in a risk framework, please describe and
provide examples of how they should be represented.
CDRH seeks input on how to account for the spectrum of GenAI-enabled informational
functions that vary in the degree to which they direct a user to a particular action (i.e.,
between “non-directive” and “action-directing”). What characteristics of an output—such as
its wording, specificity, personalization, or context—could be considered as modifiers of
the risk associated with an informational function, after accounting for the device’s
overall functionality and intended use? What additional information would provide
manufacturers with sufficient clarity and predictability around risk assessment for
informational functions while recognizing that directiveness may exist along a continuum
rather than as a binary distinction?

CDRH seeks input on whether and when a GenAI-enabled function that results in delivery of
clinical information to patients, as opposed to HCPs, could present different or higher
risks, while also recognizing the potential benefits associated with improved patient
empowerment, engagement, and access to clinical information. What device characteristics,
output features, or safeguards might mitigate risks that could arise when a user lacks the
domain knowledge to independently evaluate an output, without unnecessarily underestimating
patient capability?

CDRH seeks input on whether and how the distinction between generalist and specialist
physicians might inform the assessment of risk for HCP-facing GenAI-enabled functions.
Under what circumstances, if any, might risk be affected when an HCP who lacks the relevant
clinical specialist knowledge receives information that falls within a specialist area of
practice? What device characteristics or safeguards might mitigate such risks?

For multi-turn conversational GenAI-enabled devices that may migrate from providing “nondirective” information to “action-directing” information over the course of an exchange, how
should risk be assessed across realistic conversational trajectories? How could the
intended use of such a device be characterized when its behavior is emergent across a
conversation?
For GenAI-enabled care escalation functions, CDRH is considering how the evaluation may
account for both under-escalation and over-escalation. How could manufacturers characterize
and weigh these two directions of error, given that they may not be commensurable and that
acceptable trade-offs may vary by clinical context?
Section V: A Competency-Based Approach for Premarket Evaluatio

Is the competency-based approach described above, i.e., device benchmarking followed by
clinical confirmation, a useful and appropriate framework for evaluating GenAI-enabled
devices?

How might the two-axis risk framework described in Section IV be considered within the
competency-based approach described above to help determine the level of evidence needed
for a premarket submission?

Would a benchmarking structure such as the one described above be likely to provide
adequate evidence of clinical knowledge, analytic capabilities, safety behavior,
communication, and generalizability to support a reasonable assurance of safety and
effectiveness? Are there elements that are missing, redundant, or inappropriately
categorized? Are there externally developed standards that could be leveraged?

CDRH recognizes that publicly available benchmarking assets may be subject to data
contamination, saturation, and limited real-world representativeness. How should a sponsor
establish that performance on a given benchmark predicts safe and effective real-world
behavior for the device’s intended use? What evidence should support the construct validity
of a benchmark used to gate device evaluation, and what role should sponsor-developed
benchmarks play given potential concerns around independence and optimization to the test?
CDRH is considering that clinical confirmation for a GenAI-enabled device might not require
a prospective clinical study in every case, and has described above a range of approaches
of increasing rigor and patient exposure. How might a sponsor select and justify a
confirmation approach tailored to a device’s intended use and proportionate to the device’s
risk profile? Are there device types or risk profiles for which one or more of these
approaches would be insufficient or inappropriate? How might the anticipated distribution
of real-world inputs be taken into consideration? Are there other methods of clinical
confirmation that might help inform the evaluation of GenAI-enabled devices?
How should sponsors achieve statistically meaningful performance measurement for GenAIenabled devices? Where synthetically generated inputs supplement real patient data, how
should sponsors account for differences between the synthetic and real-world distributions
when estimating performance, and under what conditions, if any, is it appropriate to
combine benchmarking evidence and clinical confirmation evidence to support a single
performance estimate?

For which clinical domains, device functions, or subpopulations is synthetic data
particularly well-suited, or particularly inadequate, as a supplement to realworld
evidence? What safeguards would mitigate the risk that synthetic data generated by models
of the same class as the device under evaluation reproduces the very performance gaps the
evaluation is intended to detect, particularly for underrepresented subgroups?

For open-ended device outputs, how should performance comparators and acceptance criteria
be selected? When a panel of qualified clinicians serves as the comparator, how should the
applicable standard (for example, the standard of care versus the performance of a median
clinician in practice) be defined and justified? Should generalist or specialist physicians
be used as a performance standard? When should human-AI team performance, rather than the
device operating alone, serve as the basis for evaluation?
Are there ways in which performance might be assessed relative to the care, technology, or
course of action likely to occur in the absence of the device, rather than to the
comparators described in this section? What approaches might be used to identify and
justify a comparator such as unaided clinical judgment, delayed specialist review, or no
intervention?
Should independent third parties be involved in some or all aspects of a competency-based
approach, including device benchmarking and clinical confirmation? If so, in what ways
might qualified, independent third-party participation contribute to this approach, and
what qualifications and independence criteria should apply? Are there aspects of the
assessment for which third-party involvement would be impractical or inadvisable? What
safeguards or program design features would be critical to prevent third-party
participation from limiting competition or preventing innovation?

CDRH is exploring whether a competency-based approach could be applied to devices that
incorporate a variety of underlying model architectures, such as multimodal vision-language
models and generative or predictive world models. Are there aspects of the described
approach that might be ineffective or inapplicable to these devices? What additional or
distinct considerations might need to be addressed?
Section VI: Postmarket Monitoring for GenAI-Enab
CDRH is considering whether it may be appropriate to accept greater premarket uncertainty
regarding a GenAI-enabled device’s benefit-risk profile through greater reliance on
postmarket monitoring. Under what conditions might such an approach be appropriate, and
what characteristics of a monitoring program would need to be in place to justify reduced
premarket evidence? Are there device types or risk profiles for which this approach would
not be appropriate?
Please comment on the potential approaches to postmarket performance evaluation, including
periodic re-benchmarking, sample-based clinician review, and performance degradation
monitoring. What additional approaches should CDRH consider, and how should the cadence and
triggering events
Please comment on for reassessment
whether be determined?
the proposed approaches to postmarket monitoring can be
facilitated by machine-based supervisory agents. What considerations, including the
evaluation and reliability of the supervisory agent itself, should CDRH take into account
for such an approach?

What roles might clinicians, healthcare institutions, professional societies, standardssetting bodies, and other stakeholders appropriately play in postmarket monitoring, and how
can these roles be structured without diffusing manufacturer accountability?
CDRH envisions that a manufacturer’s premarket competency-based assessment might serve as a
baseline against which post-deployment modifications could be re-evaluated. How might the
extent of re-benchmarking or other evidence be scaled to the nature and expected impact of
a given modification? For example, are there categories of change that might not
significantly affect safety or effectiveness of a GenAI-enabled device, or might be
appropriately managed within a sponsor’s quality management system versus requiring FDA
premarket review and authorization before implementation? Are there categories of changes
that might be appropriate for inclusion in a PCCP?

How might PCCP concepts or other change-control approaches be adapted for GenAI-enabled
devices when the nature or scope of future modifications cannot be fully prespecified?
For devices built on third-party foundation models, changes to the underlying model may be
initiated by the third-party model developer rather than the device manufacturer. How can a
manufacturer detect, evaluate, and respond to such changes in a timely manner? What
mechanisms—for example, contractual, technical, or through a PCCP—could provide reasonable
assurance that third-party developerWould voluntary Foundation Model MAFs be practical for foundation modelSection VII:to
developers Other Topics
provide and to keep current, and what content might they include? Given that participation
would be voluntary and that model developers may have limited incentive to disclose safetyrelevant information, what would make such a program sufficiently useful for premarket
review? Are there alternative mechanisms CDRH should consider for obtaining information
about underlying foundation models?

Are there additional considerations that inform the premarket and postmarket evaluation of
agentic GenAI-enabled devices, beyond those applicable to non-agentic GenAI-enabled
devices? How should the elevated risk associated with autonomous multi-step action, tool
use, and reduced opportunity for human review be reflected in acceptance criteria and
oversight?
 Proposed Feedback
ssment of Risk for GenAI-Enabled Devices
Arthrex supports the two-axis framework as an organizing heuristic, with
three refinements. First, FDA should map each risk region to existing
regulatory constructs, including Basic or Enhanced Documentation Levels,
IEC 62304 software safety classifications, and postmarket monitoring
expectations. Second, the framework should account for downstream
safeguards, intended users, reversibility, and source traceability;
clinician review before patient action can materially reduce risk relative
to autonomous intraoperative action. Third, clinician-reviewed
informational functions should be consolidated into a limited-risk
informational category, while patient-controlled functions may warrant
additional controls.
If FDA retains separate non-directive and action-directing categories,
CDRH should establish objective boundaries among non-directive, actiondirecting, and action-taking functions. Action-taking should mean that the
device initiates or executes an action without prior clinician oversight;
clinician-reviewed recommendations should remain limited-risk
informational functions. Risk modifiers should focus on the intended user,
clinical context, personalization, severity of potential harm, and whether
a downstream clinical workflow occurs before action, rather than
subjective wording alone.
Patient-facing functions may present different risks from HCP-facing
functions, particularly when marketed directly to patients rather than
prescribed by a clinician. These risks can often be mitigated through
persistent disclosure that outputs are AI-generated and may contain
errors, instructions to seek professional medical advice, appropriate
interface controls, bidirectional intended-use traceability, and auditable
records of the basis for and resulting recommendations, rather than
automatically requiring greater premarket evidence.
CDRH should assess device risk against the intended user population
specified in labeling rather than distinguishing among individual
generalist and specialist physicians, because training, credentialing,
privileging, and off-label clinical use are matters of medical practice.
When a labeled intended use depends on specialist knowledge, appropriate
safeguards may include specialist review, intended-use traceability,
comprehensive audit trails, and periodic specialist review of operational
data, without restricting lawful clinician discretion.
For clinician-facing conversational devices, progression from nondirective to action-directing content should remain limited risk when
clinician review occurs before action. The intended use should be defined
by the highest-risk function the device is designed to perform, with
validated scope-maintenance guardrails and comprehensive audit trails
addressing context and memory. Because possible conversations are
effectively unbounded, any required realistic trajectories and validation
methods should be defined by FDA or an FDA-recognized consensus standard
rather than left to exhaustive sponsor mapping.
 Under-escalation and over-escalation should be evaluated as separate
hazardous situations within the existing ISO 14971 risk-management
process, using distinct severity, probability, and acceptance criteria
appropriate to the clinical context. Sponsors should justify measures such
as escalation sensitivity and specificity and use routine operational
monitoring and audit review to detect emerging imbalance or performance
degradation; a separate risk framework is unnecessary.
Premarket Evaluation of GenAI-Enabled Devices
As currently described, the competency-based approach is theoretical and
insufficiently connected to established design-control and softwarelifecycle processes. Benchmarks should address both competency and data
performance, be expressed as traceable software requirements and risk
controls, and be verified through defined test cases. FDA should provide
examples, protocols, scoring rubrics, and acceptance criteria and clarify
how fixed third-party and sponsor-defined tests will remain valid as
technologies evolve and how they should appear in initial and subsequent
submissions.
The GenAI risk classification—not the competency-based approach—should
determine the level of premarket evidence. FDA should link the two-axis
framework directly to IEC 62304 safety classifications, software
Documentation Levels, postmarket monitoring expectations, and residual
risk after accounting for downstream clinical workflow controls, supported
by traceable intended-use mappings and maintained benchmark baselines.
The proposed benchmarking structure does not yet provide adequate evidence
of clinical knowledge, analytical capability, safety behavior,
communication, or generalizability because its elements and criteria
remain ambiguous. An FDA-recognized consensus standard should define
benchmark methods, calibration data, acceptance criteria, and adjudicator
or LLM-grader reliability, while leveraging existing standards such as ISO
14971 and AAMI TIR34971 rather than duplicating them.
FDA should maintain a sequestered, periodically refreshed calibration set
with common acceptance criteria to reduce contamination and saturation.
Sponsor-developed and domain-specific benchmarks should supplement, not
replace, that baseline and should demonstrate construct validity through
representative clinical-use or field-use evidence, clear traceability to
software requirements and risk-control measures, and controls against
optimization to the test; validated sponsor benchmarks may subsequently
inform broader industry standards.
Clinical confirmation should be proportionate to the two-axis risk profile
and consistent with the established Class I, II, and III risk-based
framework. Prospective studies should generally be reserved for severerisk, fully autonomous action-taking functions without clinician review
before potential harm, while limited- and moderate-risk functions may be
supported by suitable alternative evidence and robust postmarket
monitoring when they do not directly affect patient safety or clinical
decision-making.
 Synthetic inputs should be permitted to supplement real patient data,
particularly where real-world data are initially scarce, when sponsors
document the generation method, representativeness, and statistically
meaningful evaluation process. Unless equivalence between synthetic and
real-world distributions is demonstrated, performance should be reported
separately; benchmarking and clinical-confirmation evidence should
likewise remain separate for safety-critical functions and should be
assessed against an appropriate common calibration framework.
Synthetic data are particularly well suited to robustness and safetybehavior testing, including adversarial prompts, paraphrase consistency,
out-of-scope requests, rare safety-critical presentations, and
parameterized anatomical or imaging variation. Safeguards should include
generation independent of the device under evaluation, clinician
plausibility review, documented generation parameters, targeted real-world
validation—including underrepresented groups—separate reporting of real
and synthetic results, and ongoing audit-based monitoring under a
consensus standard.
FDA should define performance comparators and acceptance criteria through
a common calibration set and consensus standard. The comparator population
should match the intended users identified in labeling and may include
clinicians with differing experience when justified by the domain and risk
profile. When clinician review precedes action, human–AI team performance
should be the primary basis for evaluation because it reflects actual use
and recognizes the clinician as a risk control.
Performance should be assessed against the care, technology, or action
reasonably expected without the device, using a comparator justified by
the intended use and clinical context. Representative retrospective realworld evaluation may be appropriate for limited- and moderate-risk
devices, whereas severe-risk functions should generally require
prospective comparison.
Independent third-party participation should be required only when
justified by severe device risk. Qualification and independence criteria
should preserve competition and innovation, while recognizing that a
medical-device manufacturer with dedicated, controlled resources may be
capable of conducting an objective assessment; third-party involvement
should not become a universal barrier to development or market entry.
Competency evaluation should be architecture- and modality-aware. For
multimodal systems, sponsors should assess each modality separately, the
relative contribution and interaction of modalities, and the accuracy and
safety of the combined output. User qualifications, labeling, benchmark
design, and acceptance criteria should reflect the increased interpretive
complexity as the number of supported modalities grows.
ring for GenAI-Enabled Devices
Greater reliance on postmarket monitoring is appropriate for GenAI-enabled
devices that are not severe risk, provided the monitoring program can
reliably detect degradation and support timely investigation and
corrective action. Severe-risk devices should continue to require stronger
premarket evidence and should not rely primarily on postmarket controls to
resolve material uncertainty.
 Clinicians, healthcare institutions, professional societies, standardssetting bodies, and other stakeholders should contribute domain expertise
and assist in interpreting postmarket performance because GenAI outputs
may require contextual judgment. Their roles should be defined through
clear review, reporting, and escalation responsibilities without diffusing
the manufacturer's ultimate accountability.

A PCCP should distinguish between the device harness and the foundation
model and establish controls that permit each component to change
independently. The plan should define component-specific modification
protocols, evaluation methods, acceptance criteria, and boundaries for
changes managed within the quality system versus those requiring
additional FDA review.
Manufacturers should own and control the device harness used to detect,
evaluate, and respond to third-party foundation-model changes. FDA should
clarify whether the harness is treated as a model or as software, and
contractual, technical, and PCCP controls should separately identify and
assess changes to the harness and the underlying model.
Other Topics
A voluntary Foundation Model MAF would be useful only if developers keep
it current and transparently disclose training-data sources, training
methods, and controls governing training and model behavior. The program
should provide sufficiently reliable, review-relevant information to
offset the limited incentive for voluntary disclosure.
Agentic devices require explicit monitoring for response drift in addition
to data drift. Their autonomous multi-step actions, tool use, and reduced
opportunity for human review should be reflected in more stringent
acceptance criteria, comprehensive auditability, and periodic review by
qualified domain experts.