UL Solutions (Derek Greenauer)

Please see attached

RE: Considerations for the Regulation of Generative AI-Enabled Medical Devices:
Discussion Paper and Request for Feedback; Docket: FDA-2026-N-7874

October 19, 2026

UL Solutions supports FDA’s efforts to develop a competency-based framework for
evaluating generative AI-enabled (GenAI-enabled) medical devices and appreciates the
opportunity to provide comments on the discussion paper. GenAI-enabled medical devices
hold significant potential to improve patient care, while also presenting distinct evaluation
challenges. A competency-based framework offers a way of evaluating whether a device
can reliably perform its intended functions across a range of inputs, conditions, and
scenarios, while a lifecycle approach can help ensure performance is maintained as the
device and its underlying components change over time. As described in our comments,
the discussion paper’s proposed risk assessment framework, benchmarking and evidence
considerations, lifecycle assurance approach, and treatment of foundation models provide
a strong foundation for building confidence in these technologies. We look forward to
continuing to work with FDA and other stakeholders to establish appropriate, riskproportionate evidence and evaluation approaches across the medical device ecosystem.

UL Solutions is a global safety science company with extensive experience in the testing,
certification, and conformity assessment of complex technologies, including medical
devices and AI-enabled products. Our medical device work spans safety and performance,
software, cybersecurity, and quality management, including testing within FDA’s
Accreditation Scheme for Conformity Assessment (ASCA). This experience informs our
perspective on how FDA can develop a rigorous and risk-proportionate regulatory approach
for GenAI-enabled medical devices. In our responses below, we have focused on the
questions where we have relevant expertise.

Risk Assessment (Questions 1, 2, 3, and 5)

UL Solutions supports FDA’s proposed two-axis framework based on device activity and
the consequences of relying on an incorrect output, though we suggest some additional
factors that may inform the appropriate premarket and postmarket evidentiary
expectations. Where practicable, these risk characteristics should be defined in a manner
that supports consistent, risk-proportionate assessment and testing and can inform future
standardization.
Consequence modifiers. This framework should consider factors that affect how likely an
erroneous output would be to lead to harm. Errors that go undetected or those for output
that cannot be easily traced to underlying evidence may be more likely to lead to harm, as
would output that does not clearly convey uncertainty, or output that leads to harms that
cannot be easily reversed. Therefore, error detectability, evidence traceability, uncertainty
communication, and reversibility of harm should be considered as potential additional risk
modifiers.

Activity modifiers. FDA should also consider factors that influence the practical level of
independence exercised by the system. These include opportunities for meaningful human
intervention, user expertise, and foreseeable user reliance on outputs. Human involvement
should only be considered to reduce risk when the user possesses the expertise, time,
information, and authority necessary to evaluate and override the system’s output.
Likewise, highly personalized, confident, or workflow-integrated outputs may function as
de facto recommendations regardless of how they are labeled. These considerations
modify the degree to which a system should be viewed as informational, action-directing,
or action-taking.

Competence and human interaction. Initial and ongoing competence should be
addressed for organizations and personnel performing development, validation,
assessment, oversight, and other safety-critical activities. Assessment should also
consider safety-relevant behavior that may emerge across realistic, multi-turn human-AI
interactions.

Cross-cutting system factors. The risk of a GenAI-enabled device also depends on its
deployment context and external dependencies. Foundation models, retrieval systems,
external data sources, integrations, or other components outside the manufacturer's direct
control may affect both device activity and consequence severity.

Scale and propagation of harm. FDA should also consider the potential scale and
propagation of harm, as the same erroneous output may present different risks depending
on whether its effects remain isolated or can propagate across patients, workflows, or
connected systems. The potential for error propagation, cascading failures, and multipatient harm should therefore serve as additional modifiers of overall risk and
corresponding evidence requirements.

Benchmark Governance, Validation, and Evidence (Questions 7, 8, 9, 10, and 16)

Benchmark development, maintenance, and application should be guided by transparent
governance standards that address factors like representativeness, contamination,
saturation, and optimization to known test assets. In addition, FDA and the stakeholder
community should advance toward benchmarks that are:
• Valid and clinically relevant, with demonstrated construct validity and predictive
value.
 • Risk-based and representative, with test scenarios aligned with intended use.
• Periodically reassessed as technologies, intended uses, and clinical practices
evolve.
• Supported by test methods, evaluation criteria, acceptance criteria, and resultinterpretation guidance that enable reproducible assessments.
• Supported by assessment methods that are, where practicable, architectureneutral and applicable across evolving AI architectures and capabilities.
• Adapted for open-ended GenAI outputs through consistent scoring methods that
account for variability and multiple acceptable responses, with synthetic data used
to supplement representative real-world data and support sufficiently extensive
testing.
• Consider the full technology stack, including computing infrastructure, integrations,
and safety guardrails.

Role of Third Parties

Evaluation by qualified independent third parties supports confidence, consistency, and
transparency for FDA’s regulation of GenAI-enabled medical devices, building trust in
manufacturers’ performance and safety claims among regulators, healthcare providers,
and patients. Qualified third parties can:
• Improve consistency and comparability by applying common evaluation
methodologies and criteria across manufacturers and assessment cycles.
• Promote transparency through documented procedures, evidence, and reporting
that support traceable evaluation conclusions.
• Provide an independent perspective, particularly for higher-risk applications and the
assessment of safety-relevant claims and potential failure modes.
• Apply specialized expertise to novel technologies and complex evaluation
methodologies, including cybersecurity, interoperability, and human factors.

Third parties could also contribute to benchmark development, validation, governance,
testing, and periodic review. Their involvement should provide an independent perspective
and support reproducible and consistently applied assessment methodologies.

FDA could draw on its existing experience recognizing qualified laboratories through
programs such as the Accreditation Scheme for Conformity Assessment (ASCA) as it
considers approaches to benchmark evaluation for GenAI-enabled medical devices.
Standards-based accreditation helps promote the competence, impartiality, and
consistent performance of third-party evaluators. Over time, consensus standards and
relevant accreditation frameworks, like ISO/IEC 17025, could support a competitive
ecosystem of qualified organizations capable of providing reliable benchmark evaluation
services to GenAI-enabled device manufacturers.
Integrate organizational AI governance. FDA should encourage manufacturers to
integrate ISO/IEC 42001 certification into their existing ISO 13485 quality management
systems so that device-level testing is supported by organization-wide controls for AI risk,
data, change, and performance management. This could also allow for the clear and
traceable articulation of safety-, security-, and sustainability-related metadata into the
model and data cards of regulated AI systems.

Support reliable assurance information. Where model cards, data cards, system cards,
or similar documentation support benchmark selection, risk assessment, regulatory
review, or ongoing monitoring, mechanisms including third-party verification should
support confidence in their completeness, accuracy, and currency.

Life Cycle Assurance and Change Management (Questions 19, 21, 22, 23, and 24)

Postmarket assurance for GenAI-enabled medical devices should extend the premarket
evaluation across the total product life cycle. FDA should promote a structured, risk-based
framework that uses the premarket competency assessment as a baseline, defines clear
monitoring and reassessment expectations, preserves manufacturer accountability, and
addresses changes to both the device and its underlying third-party components. This
framework should:
• Connect premarket assessment with postmarket reassessment. Premarket
competency assessments should establish a defined baseline for postmarket
monitoring, periodic reassessment, and reassessment following relevant changes.
• Establish structured, risk-based monitoring. Monitoring programs should use
defined performance indicators, review cadences, and triggering events to detect
drift, degradation, and emerging risks. The frequency and rigor of reassessment
should reflect the device’s intended use, risk profile, real-world performance, and
the nature and potential safety impact of a change. Monitoring should use
appropriate, fit-for-purpose real-world evidence where relevant.
• Validate automated supervisory tools. Where AI-based or other automated tools
are used to support postmarket monitoring, they should themselves be validated for
their intended monitoring function, including their reliability, limitations, and ability
to detect relevant failures.
• Maintain quality-system assurance across deployment. Ongoing certification or
other independent assessment of the quality management systems used by
manufacturers and, where appropriate, deploying organizations, should
complement device-level monitoring and reinforce continuing compliance and
accountability.
• Scale change-control requirements to potential impact. Changes unlikely to
affect safety or effectiveness may be managed within the manufacturer’s quality
management system, while changes that could materially affect intended use,
safety-critical behavior, or previously demonstrated performance should require
 more extensive re-benchmarking, supporting evidence, or FDA review, as
appropriate.
• Use Predetermined Change Control Plans (PCCPs) where appropriate. PCCPs
can provide a structured mechanism for managing foreseeable categories of change
while maintaining appropriate risk-based controls and reassessment. FDA could
further support their application to GenAI-enabled devices by providing examples of
foreseeable categories of change.
• Address changes to third-party foundation models and external AI components.
Dependencies on foundation models and other external AI components should be
documented throughout the device lifecycle. Changes to external components
should trigger an impact assessment where safety, effectiveness, or previously
demonstrated capabilities may be affected.
• Advance standardized reassessment methods. Stakeholders and the FDA should
develop consensus standards and common methodologies for postmarket
monitoring, re-benchmarking, change-impact assessment, and evaluation of
externally initiated changes to improve consistency, reproducibility, and
comparability across the device ecosystem.

Further Ecosystem Considerations (25 and 26)

FDA should promote an ecosystem-level assurance framework that improves access to
safety-relevant information about third-party foundation models and addresses the distinct
risks presented by agentic GenAI-enabled medical devices. The framework should support
consistent regulatory review without treating the underlying model as an authorized
medical device or prescribing specific technical implementations. FDA and stakeholders
should:
• Establish a standardized minimum information set for foundation models.
Foundation Model Master Files (MAFs), system cards, or equivalent mechanisms
should provide reliable and current safety-relevant information, such as intended
uses, known limitations, healthcare-relevant evaluation results, and significant
updates. FDA and stakeholders should consider the specific contents of this
minimum information set as an area for future consensus-based standardization.
• Make voluntary information-sharing mechanisms useful and sustainable.
Foundation Model MAFs or equivalent mechanisms should support reliable access
to current safety-relevant information, including updates following relevant model
changes.
• Preserve device-manufacturer accountability. Access to a Foundation Model
MAF or equivalent information should support, but not replace, the manufacturer’s
responsibility to evaluate the final deployed device for its intended use and
demonstrate its safety and effectiveness.
• Apply system-level evaluation to agentic AI. Evaluation should address the
complete sequence of autonomous planning, tool use, external interactions, and
actions, rather than assessing individual model outputs in isolation.
 • Evaluate action boundaries and failure propagation. Test methods and
acceptance criteria should assess whether the device remains within its intended
use and safety envelope, appropriately handles erroneous tool outputs and system
failures, prevents unsafe cascading actions, and recovers safely when a task cannot
be completed as intended.
• Verify meaningful human oversight. Where human oversight is relied upon as a
risk control, evaluation should verify its effectiveness, including whether users have
the expertise, information, time, and authority needed to identify and override
unsafe actions.
• Advance standards-based assurance approaches. Consensus standards should
define assessable outcomes, test methods, acceptance criteria, and safeguards for
agentic systems while remaining technology-neutral and avoiding requirements that
prescribe a particular model architecture or technical implementation.

UL Solutions appreciates FDA’s leadership in developing a competency-based, lifecycle
approach to the oversight of GenAI-enabled medical devices. We look forward to working
with FDA and other stakeholders to advance risk-proportionate evidence expectations,
consistent and reproducible evaluation methods, and standards-based assurance
approaches that support innovation while maintaining confidence in device safety and
effectiveness. Drawing on our experience in testing, certification, conformity assessment,
and AI governance, UL Solutions stands ready to support FDA as this regulatory framework
continues to evolve. If we can provide further information, please contact Meghan
Housewright (meghan.housewright@ul.com), Government Affairs Lead.

Sincerely,

/s/
Derek Greenauer
Director, Global Government Affairs—Americas
UL Solutions