Nathan Sabich

FDA Question 23: How might PCCP concepts or other change-control approaches be adapted for GenAI-enabled devices when the nature or scope of future modifications cannot be fully prespecified?

Opinion: The PCCP Framework requires pre-specification of modifications, but GenAI foundation models evolve in ways that cannot be fully predicted. The solution is not to abandon pre-specification but to pre-specify at a higher-level of abstraction. Rather than specifying “the model will be retrained with X additional images,” the PCCP for a GenAI-enabled device should specify that the foundation model may be updated provided the updated model meets the following competency benchmarks, passes the following safety evaluations, and does not exceed the following performance deviation bounds from the authorized baseline.

Architecture to Address This: I have already created for an architecture’s governance framework that operates at exactly this level of abstraction. The AI lifecycle management controls apply across all the steps I created in my architecture in order to provide continuous QMS oversight (design controls, CAPA, change management, documentation), risk management (ISO 14971 aligned), human oversight (HITL throughout with escalation paths), cybersecurity (secure by design, threat monitoring, incident response), and transparency and labeling (intended use, limitations, performance, updates). This governance layer is change-agnostic and it applies regardless of whether the modification is a retraining event, a foundation model version update, or a prompt engineering adjustment. The key is that the governance controls remain constant even as the specific modifications vary.

Recommendation to FDA: Create a new PCCP category called "Performance-Bounded PCCP" for GenAI-enabled devices. Unlike a traditional PCCP that pre-specifies the exact modifications, a "Performance-Bounded PCCP" would pre-specify: (1) the competency benchmarks the device must continue to meet after any modification, (2) the maximum acceptable performance deviation from the authorized baseline across each benchmark dimension, (3) the monitoring methodology for detecting deviations, (4) the rollback protocol if deviations are detected, and (5) the cumulative impact tracking methodology. Any modification that keeps the device within the performance bounds may be implemented without a new submission. Any modification that causes the device to fall outside the bounds triggers either a rollback or a new submission. This approach accommodates the unpredictability of GenAI evolution while maintaining the principle that FDA authorizes the safety and effectiveness envelope, not the specific technical implementation.