Lara Zibners

I make these comments as a board-certified emergency pediatrician, a medical educator and co-founder and chairman of a women’s health care startup. I hold an MD, a MMEd and an MBA and serve on the board of the Center of the Business of Health at UNC Kenan-Flagler Business School. These comments are my own and are offered in a personal capacity.
I wish to make four core points.
Models not trained on women and children
Women have been systematically under-represented in clinical research and women’s physiology, symptomatology and disease trajectories are poorly captured in the literature, guidelines and electronic records that generative models are trained upon. Generative AI devices built upon that baseline can be both confident and wrong. The person receiving the output cannot reliably tell the difference.
An important third dimension to the risk framework needs adding. Any function that is intended for use in populations that are underrepresented in the training has to be held to a higher bar.
It is also essential that we do not allow patient-facing materials widen the gap for women, children and minorities in healthcare. When a user is not able to spot an output that reflects an adult male-default presentation, the risk tier should reflect that.
Clinician bias carried into the model
The competency-based approach is explicitly modeled on how clinicians are evaluated. That could appear to be a reasonable analogy, but it imports a significant problem that the Paper fails to address. The standard it benchmarks against itself is biased.
This is particularly true for children, who may receive inappropriate care from clinicians without specialized pediatric training. Children are not small adults and their presentation and disease progression can vary by age. Meaning small children are not the same as big children. The risk for error in our most vulnerable population cannot be tolerated.
Therefore, I would urge the panel to consider the following:
Training data that includes clinical notes and diagnosis codes reflect who was believed, investigated and treated. Not the correct diagnosis or outcome.
Guidelines that are derived from primarily male or adult populations will not demonstrate the same proof of correctness for women and children.
The median clinician as a performance standard holds particular risk for specialized populations such as children. The intended-use population should be a consideration when choosing clinicians for adjudication.
Transparency about who the data represents
The Paper proposes evaluating the final deployed device rather than the foundation model alone. I support that but there remains a gap. A sponsor can be held to account for subgroup performance without any way to know whether the underlying model considered the subgroup.
As a startup founder, I know how difficult it can be for early-stage companies to exert any leverage over larger, established entities. That is the very reason why model providers should be obligated to provide this information and not to let it rest on voluntary goodwill.
Users should also receive a plain-language model facts label. Both clinicians and patients should be able to see, at its point of use, which populations the device was validated for and where the evidence is limited.
Care escalation as a consideration
Finally, as a clinician who has practiced pediatric emergency medicine, where the death of a child is a sad yet not rare occurrence, I must draw attention to the need for special consideration of care escalation. The Paper is correct that both under and over escalation are real concerns. From an emergency and trauma viewpoint, I would add that these two errors are rarely symmetric. Under escalation for a woman with ’atypical’ cardiac symptoms or a parent with a febrile newborn is the type of failure that costs lives. These are exactly the presentations where a model trained on a majority-population data is most likely to be falsely reassuring.
Children in particular should be treated as a distinct population, not a sub-analysis. Children are not small adults. Normal vital signs vary by age. Drug dosing is dependent on age and weight. Presentation, such as sepsis or traumatic shock can differ due to pediatric physiology. I would ask that pediatric considerations be a mandatory test domain for any device whose intended use could include children or their parents. The Paper’s own parent-and-sore-throat example illustrates why. The user is a proxy who cannot independently assess the output.
In closing, generative AI could meaningfully improve access to care, including for groups that the health system has historically served poorly. However, it can only do so if the evidence behind each device reflects the people who will use it.
I would welcome the opportunity to discuss these comments further and to contribute to any follow-up work by the CDRH.
Respectfully,
Lara Zibners MD MMEd MBA
Co-founder, Chair Calla Lily Clinical Care