QRx Partners

QRx Partners appreciates the opportunity to comment on FDA’s Considerations for the Regulation of Generative AI-Enabled Medical Devices. We support a risk-based, total product life cycle approach that builds on established medical device principles. We offer comments on Questions 1, 3, 11, and 24.

Question 1: Two-Axis Risk Framework

The proposed framework based on device activity and the consequence of relying on an incorrect output is useful for initial risk characterization, but it should not substitute for device-level risk analysis.

Risk also depends on how an erroneous output could progress to a hazardous situation. Relevant factors may include the user’s ability to recognize the error, opportunity for intervention, reversibility, time to respond, and downstream safeguards.

FDA should treat the two-axis framework as a screening or organizing tool rather than a classification matrix. Further evaluation should consider the GenAI-enabled function within its intended use environment and overall risk management process.

Question 3: Patient-Facing Functions and User Understanding

FDA appropriately recognizes that a user’s ability to independently evaluate an output can affect risk. Patient-facing use, however, should not by itself create a presumption of higher risk.

FDA should consider whether a GenAI-enabled device’s ability to assess and adapt to a user’s demonstrated level of understanding may serve as a risk control. A system could adjust the specificity, explanation, uncertainty communication, or directiveness of its output based on whether the user demonstrates adequate understanding of the information and its limitations. Similar considerations may apply to generalist versus specialist HCPs.

We do not recommend assessment of user understanding as a universal requirement. Rather, user comprehension and adaptive communication should be considered potential risk controls when safe use depends on the user’s ability to understand or independently evaluate the output.

Question 11: Clinical Confirmation

We support FDA’s proposal that clinical confirmation be proportionate to intended use and risk and not necessarily require a prospective clinical study.

The key question should be what uncertainty regarding safety and effectiveness remains after benchmarking and nonclinical evaluation, and what evidence is needed to address that uncertainty. Retrospective evaluation, shadow deployment, standardized patient interactions, clinician adjudication, and prospective investigation should be considered alternative or complementary methods rather than a fixed hierarchy.

This approach supports least burdensome principles while allowing greater rigor when intended use, autonomy, potential consequences, or residual uncertainty warrant additional evidence.

Question 24: Third-Party Foundation Models and Change Control

The challenge presented by third-party foundation models is significant, but the underlying regulatory problem is not entirely novel. Medical device manufacturers already incorporate software components they did not develop and over which they may have limited development information or control.

FDA should consider building on established principles for managing Software of Unknown Provenance (SOUP), including IEC 62304 and existing FDA software lifecycle expectations. Manufacturers should evaluate the third-party component within the finished device, identify how reasonably foreseeable failures or changes could contribute to hazardous situations, evaluate resulting risks, and establish appropriate controls.

The objective should not be complete visibility into or control over the foundation model, but sufficient information and controls to identify, evaluate, and respond to changes that could affect device safety or effectiveness. Controls could include supplier change notification, model version control, regression benchmarking, behavioral monitoring, guardrails, and other device-level controls.

Foundation models introduce an additional challenge because a remotely hosted model may change without an identifiable modification to the manufacturer’s software. Mechanisms may therefore be needed to detect significant behavioral or performance changes when supplier notification is unavailable.

Consistent with FDA’s emphasis on evaluating the final user-facing device rather than the foundation model in isolation, expectations should focus on the manufacturer’s ability to establish and maintain reasonable assurance of safety and effectiveness of the finished device.

Overall Consideration

GenAI creates new failure mechanisms and assurance challenges, but these do not necessarily require replacing established risk-based medical device principles. New approaches should remain focused on intended use, reasonably foreseeable failure, residual risk, and clinically meaningful evidence rather than attempting to evaluate every capability or possible behavior of an underlying generative model.