Elgaraali Rodwan

I write as a pediatrician by training (MBBCh, MSc in Immunology, Trinity College Dublin, and Postgraduate Diploma in Paediatrics, Royal College of Physicians of Ireland) who now reviews clinical AI products for safety through DigitalProved, an independent healthcare-AI review publication. I support the competency-based direction of this discussion paper, and I offer five points from pediatric practice and product review.

1. Assess the deployed device, not the model alone. I strongly agree that benchmark results for a foundation model cannot establish the competency of a downstream device. In reviews, the failures that matter are usually produced by the whole configuration—prompting, retrieval, guardrails, workflow design, and interface—not by the base model in isolation.

2. Set minimum thresholds per safety domain, not only aggregate scores. A device should not be able to pass overall while quietly failing a safety-critical category. In particular, observation-dependent cases, where an output is fluent and complete-sounding but omits the one finding that changed the clinical decision, should be benchmarked as their own category, tested against the source record, because tidy wording makes such omissions harder, not easier, for a clinician to spot.

3. Make pediatric performance explicit. Children are not a generalization problem that averages out. Age-specific vital-sign norms, weight-based dosing, developmental differences, and caregiver-mediated use mean a device can perform well in adults and unsafely in children. Where a device’s intended use includes or foreseeably reaches pediatric patients, benchmarking and clinical confirmation should be stratified by age band, include dosing-recalculation cases with verified weights, and test caregiver-facing outputs for misunderstanding risk.

4. Treat every upstream model change as a re-benchmarking trigger. Periodic re-benchmarking and sample-based clinician review are the right postmarket tools, but they should be triggered by any change to the underlying foundation model or deployment architecture—including silent upstream updates—against thresholds declared in advance, not only on a calendar schedule.

5. Be cautious with synthetic data in pediatrics. Synthetic cases and virtual patients are useful for standardization, but they systematically under-represent rare presentations and atypical trajectories, which are exactly where pediatric harm concentrates. Sponsors relying on synthetic data should be required to state how the distribution gap against real-world pediatric data was measured and addressed.

Finally, I support the proposal for qualified third parties to hold sequestered datasets and provide independent clinician adjudication. Independent, reproducible evaluation—with error categories, evidence spans, and severity levels—is what will make competency claims trustworthy to the clinicians expected to rely on these devices.