Steve Lee

I am an academic otolaryngologist studying AI in ENT practice. My group works on safety benchmarks for ambient AI scribes, verification interfaces for high-risk documentation fields, shadow-deployment LLM triage comparisons, and a patient-facing assistant for dizzy patients. I write as a clinician who will use these devices and answer for their outputs. Five points:

**1. Benchmarks must be specialty-anchored.** The paper’s evaluation structure is sound, but general benchmarks miss the errors that harm patients: in our scribe work, the consequential failures are terminology errors, laterality flips, and misattributed exam findings — invisible to general evaluations. Require specialty-anchored benchmarks with clinician-built error taxonomies.

**2. Assess risk at the field level, not just the function level.** The two-axis risk framework is right, but consequence varies within a single function: one generated note has fields where an error is trivial and others where it is catastrophic — wrong surgical laterality, a wrong dose, a dropped allergy. Require sponsors to identify high-consequence fields prospectively and mitigate at the field level, not merely report aggregate scores.

**3. Recognize shadow deployment as clinical-confirmation evidence.** Between retrospective review and randomized trials sits a missing middle: prospective silent deployment — the device runs on live inputs, outputs compared against attending ground truth, no patient exposure. We are running this for ENT referral triage. Accept it, with prespecified endpoints and blinded adjudication, as valid evidence for moderate-risk functions.

**4. Tie drift responsibility to foundation-model changes.** Many clinical GenAI tools — including our dizziness assistant — are thin layers over commercial models outside the sponsor’s control. When the underlying model shifts silently, the safety profile changes with no sponsor action. Trigger re-benchmarking on model version changes, not calendar schedules; require model/version disclosure in labeling; and keep drift detection the sponsor’s responsibility even when drift originates upstream.

**5. Put patient-facing diagnostic tools at the top of the risk chart.** Our dizziness assistant interviews patients and generates an HPI and differential — high independence, catastrophic failure modes (a missed posterior circulation stroke). Such tools deserve the most rigorous confirmation, up to prospective trials — never clearance on benchmarks alone. And reckon with who explains the output: the burden falls on a clinician who never saw its reasoning. Require an auditable trace sufficient to reconstruct — and contradict — the device’s reasoning.

Discussion papers become guidance, and guidance becomes what manufacturers must demonstrate before reaching a hospital. The specialty-clinician perspective belongs in this record. Thank you for the opportunity to comment.

Respectfully,
Steve Lee, MD
Associate Professor of Otolaryngology, Loma Linda University