Grifols

We appreciate FDA’s efforts to develop a regulatory framework for GenAI-enabled medical devices. The discussion paper provides a valuable foundation, and additional clarification in the areas outlined below (see attached document) would support consistent interpretation and implementation.

Comments on Considerations for the Regulation of
Generative AI-Enabled Medical Devices

General Comments
We appreciate FDA's efforts to develop a regulatory framework for GenAI-enabled medical devices. The discussion
paper provides a valuable foundation, and additional clarification in the areas outlined below would support consistent
interpretation and implementation.

1. Section IV - Considerations for the Assessment of Risk for GenAI-Enabled Devices

The paper proposes a risk assessment framework based on activity/autonomy and the consequence of an incorrect
output. It may be useful to also incorporate detectability, similar to FMEA principles, since risk can vary significantly
depending on the ability to detect an error before it leads to a consequence. This could also be linked to Reasonably
Foreseeable Misuse (uses outside the intended purpose that can be reasonably anticipated based on human behavior).
Should detectability of an incorrect AI-generated output be considered as an additional dimension in the risk
assessment?

2. Section IV - Considerations for the Assessment of Risk for GenAI-Enabled Devices

Although the paper considers different levels of human oversight, it is not clear what level is considered sufficient or at
which point such oversight should occur. One possible approach would be to define, based on function and risk, which
activities AI may perform autonomously and which actions require a human oversight checkpoint by a Qualified User.

Could FDA provide clearer guidance on the level of human oversight required based on the risk of each AI function,
including when review is mandatory and who is qualified to perform it?

3. Section V.D.1 - Standards of Performance; R.1 - Robustness, Reliability, and Reproducibility

The paper acknowledges that, for GenAI systems, there may be more than one correct output and that some degree of
variability is expected. It would be helpful to clarify how acceptance criteria should be established when such variability
is inherent to the model, particularly for functions with different risk levels.

How should manufacturers define the acceptance criteria for GenAI-enabled device performance, considering that there
may be more than one correct answer and that the same or similar input may generate different outputs?
Should minimum performance thresholds and acceptance criteria be function-specific and risk-based, even when overall
model performance remains acceptable?

4. Section VI - Postmarket Monitoring for GenAI-Enabled Devices
The paper places greater emphasis on post-market monitoring than on attempting to cover all possible inputs and
outputs during premarket testing. In addition to the periodic use of approved test datasets, feedback from healthcare
professionals regarding the accuracy of outputs could provide valuable real-world performance evidence. For high-risk
functions, consideration could also be given to implementing a stop mechanism or temporary restriction when
performance no longer meets predefined criteria, pending review by a qualified individual.
Should post-market monitoring include mechanisms capable of automatically restricting AI functionality when predefined
performance criteria are not met, pending assessment by a Qualified User?

Page 1 of 2
Comments on Considerations for the Regulation of
Generative AI-Enabled Medical Devices

5. Section VII.A - Foundation Model Device Master Files (MAF)
When the model is provided or updated by a third party, it would be helpful to clarify the level of control expected from
the manufacturer, as well as responsibilities related to MAF maintenance and the assessment of model changes. In
addition, the MAF should provide sufficient transparency to support understanding of the model and its limitations, and
establish mechanisms for escalating manufacturer feedback to the model provider.
What level of control is expected from a medical device manufacturer when the underlying AI model is provided and
updated by a third party?

6. Clinical Confirmation Approaches
Synthetic data may be particularly valuable for rare cases and edge cases; however, it should not be automatically
assumed to be clinically valid. The clinical validity of both the generated case and the expected outcome should be
reviewed and approved by appropriately qualified experts who can verify that the case adequately represents the clinical
scenario or failure mode being evaluated, and that the dataset is appropriate for the intended function and intended use
under assessment.

When synthetically generated clinical cases are used, should the clinical validity and expected outcomes of such cases
be independently reviewed and approved by appropriately qualified clinical experts before they are accepted as part of
the reference test dataset?

Page 2 of 2