Carexdata LLC

We support the paper’s two-axis risk framing (degree of autonomy × severity of reliance on a wrong output) and the proposal of competency-based premarket evaluation that combines non-clinical benchmarking with clinical confirmation. Our comments focus on how such evaluation can be made reproducible and inspectable, drawing on experience building automated checks that run on the actual artifacts (training data, model binaries, source code, submission documents) rather than on narrative descriptions.

1. Competency-based evaluation should be machine-reproducible
Non-clinical benchmarks for generative functions are only useful as evidence if a reviewer or auditor can re-run them. We recommend FDA encourage sponsors to submit (a) the benchmark inputs or their fingerprints, (b) the exact model identifier and version, including the foundation-model version where applicable, (c) the scoring procedure as executable code or a precise specification, and (d) the environment description (SBOM-style). This mirrors what the June 27, 2025 cybersecurity guidance already expects for software composition and would let competency claims be verified rather than asserted.

2. Autonomy and severity should be tied to measurable guardrails
Many generative functions rely on runtime controls: output constraints, confidence thresholds, fallback behavior and escalation to a human. These controls are observable in code and can be tested. We actually would recommend the final framework ask sponsors to document each guardrail, the test that exercises it, and the evidence that it fires under distribution shift. Automated static and dynamic checks can produce this evidence cheaply, so the burden on small sponsors need not be high.

3. Foundation-model change is a data and version problem, not only a labeling problem
When a third-party foundation model changes, the device’s competency evidence may silently expire. Here we recommend that postmarket monitoring expectations include version pinning (or detection of unpinned dependencies), scheduled re-execution of the premarket benchmark against the deployed model, and a defined threshold for when drift triggers a review under the PCCP. Tooling to do this continuously already exists in software engineering practice (CI gates, drift monitors) and can be adapted to the clinical context.

4. Training-data provenance and subgroup coverage should remain in scope for generative functions
The paper’s questions about competency should extend to the data used for fine-tuning and evaluation: duplicate and leakage detection between tuning and evaluation sets, representativeness across the intended-use population, and residual PHI in prompts, images or fine-tuning corpora. These are automatable checks and align with the existing GMLP principles and the data-management expectations in the January 2025 lifecycle draft guidance.

5. Proportionality for small sponsors
Small developers hold a large share of AI-enabled device authorizations. We encourage FDA to publish example evidence packages for low-autonomy, low-severity generative functions so that competency evidence can be produced with modest resources, and to make clear which elements can be satisfied by automated, reproducible checks versus which require clinical confirmation.

6. Standards gap
ISO/TS 24971-2:2026 explicitly excludes LLM/generative functions. Until a standard covers them, FDA’s framework will be the de facto reference. We encourage FDA to align terminology (autonomy, severity, competency, guardrail) with ISO 14971 and ISO/TS 24971-2 so that sponsors can maintain one risk file.

Thank you for the opportunity to comment. We would be glad to share benchmark results from our automated evaluation of open-source medical AI software if useful to CDRH staff.

CarexData LLC - info@carexdata.com