How should under-escalation be weighed against over-escalation?

Question 6 — under- and over-escalation. For a bystander function the two directions are not commensurable in the way the paper describes for triage-at-home, because escalation is not a choice the function makes; it is a step in every protocol. The failure mode that matters is not "the model told them to call when they didn't need to," it is ordering: does the function put the call for help before or after the first action, and does it do so correctly by scenario (for a lone rescuer with an unresponsive child, current guidance differs from the adult case). I would suggest that for protocol-delivery functions, escalation be evaluated as a protocol-fidelity item with a scenario-specific right answer, rather than as a scalar trade-off between two error rates.

R8. Escalation errors are asymmetric and should be weighted that way (Question 6) This follows from S5. CDRH is right that under- and over-escalation "may not be commensurable." I would go further: they are asymmetric in a specific direction. Under-escalation of a time-critical condition produces irreversible harm to an identifiable patient. Over-escalation produces anxiety, cost, and system burden, which are real but recoverable. A loss function that weights these equally is not neutral; it is a policy choice that favors system efficiency over the patient in front of the device. I recommend that sponsors be expected to prespecify and justify the trade-off, that labeling disclose it in plain language, and that CDRH decline to accept arguments in which emergency department utilization costs justify a looser escalation threshold for patient-facing functions.

Question 6 — Weighting under- and over-escalation Under-escalation and over-escalation create different kinds of harm, so they should not be reduced to one overall accuracy number. If only one number is reported, the system may be tuned toward the error that is easiest or cheapest to reduce. Sponsors should explain the trade-off clearly at the start, the same way analytical method validation makes key assumptions and acceptance criteria clear before testing begins. Recommendation. Ask sponsors to state, in advance, how they will weigh under-escalation and over- escalation, and why that weighting makes clinical sense. Performance should be shown across the trade- off, not as a single best number. Sponsors should also identify the operating point they chose and explain why it fits the setting where the device will be used.

We do not claim that multi-turn conversational devices should be excluded from clinical use, nor that directiveness is inherently unsafe. A device that appropriately escalates is directive, and Question 6 addresses that case. Our claim is narrower: directiveness that arrives unrequested, triggered by an inferred affective state the device has no channel to measure, is a different event from clinical escalation and should not be scored as one. Finally, we note the limits of our own instrument. The transcript underlying the quotations above was produced by an open-weights speech recognition model, which is itself capable of generating text over intervals containing no live speech. We measured the recording for such intervals before transcribing it, in FDA-2026-N-7874 — Question 5 5 The Christman AI Project contiguous 250 ms windows across its full duration, and confirmed that no quoted passage falls within one. The audio recording, not the transcript, is the primary evidence. 6. Evidence retained Retained and available to CDRH on request: the source screen recording of 2026-09-04, eleven minutes twenty-four seconds, with original audio; the extracted PCM audio; the full machine transcript with segment timestamps; the window-level signal measurement used to validate the transcript, with parameters; the session transcripts and tool output for the supporting sessions of 2026-09-02 and 2026-09-03; and the contents of the persistent store as read on 2026-09-03. Every quotation in Section 1 carries a timestamp into the source recording and can be verified against it directly. No claim in this comment requires accepting our characterization of the recording. 7. About this submission The Christman AI Project builds augmentative and alternative communication systems for nonverbal and neurodivergent users, cognitive support for dementia care, and related assistive technology. The submitter is autistic and builds for this population directly. We file on Question 5 because the trigger identified above is one our users produce constantly and cannot suppress. Distress, frustration, flat affect, atypical prosody and long pauses are ordinary features of how the people we serve communicate, and several of them are the clinical presentation itself. A device whose directiveness rises with the user’s apparent emotional state will be at its most directive with the users least able to refuse the direction, and it will present as attentive and improving the entire time. The session recorded here was caught because the user was an engineer who had built the systems under discussion, was recording deliberately, and knew the source document did not exist. None of those conditions holds in deployment. Session recorded 2026-09-04. Supporting records 2026-09-02 to 2026-09-03. Submitted to Docket FDA-2026-N-7874, comment period closing 2026-10-19. Contact: contact@thechristmanaiproject.com Submitted by Everett N. Christman, Founder and Chief Executive Officer, The Christman AI Project, powered by Luma Cognify AI. Signature: _______________________________ Date: __________________ FDA-2026-N-7874 — Question 5 6 The Christman AI Project

Question 6: Care escalation The manufacturer must measure three types of error: Under-escalation. Over-escalation. Failed escalation. Failed escalation occurs when the device starts an escalation but the handoff does not work. The device can select the wrong service. The alert can arrive too late. The responsible person can fail to act. The manufacturer should measure: Sensitivity for required escalation. False-positive and false-negative rates. Time to escalation. The selected care destination. Completion of the handoff. The clinical and operational result. A severe under-escalation error needs its own pass or fail limit. Good performance on common cases must not compensate for a severe failure. Newton’s Tree Inc Considerations for the Regulation of Generative AI-Enabled Medical Devices FDA Docket No. FDA-2026-N-7874 Layer 1: Competency

3.3 Question 6 - Under-escalation and over-escalation The FDA recognises both under-escalation and over-escalation as clinically important failure modes (FDA, 2026a). Sitora recommends that they should never be hidden inside a single aggregate “accuracy” score. An AI that misses an urgent presentation and an AI that unnecessarily recommends a routine consultation have both erred, but the clinical consequences are not comparable. Dimension Clinical severity Probability Urgency Reversibility Safeguards Resource burden Table 3. Risk-weighted escalation matrix. Source: Sitora analysis based on FDA (2026a). Performance reporting should therefore distinguish at least critical under-escalation, non-critical under- escalation, critical over-escalation and non-critical over-escalation. In safety-critical use cases, sensitivity to serious red flags should be evaluated separately from overall system accuracy.

Response to Discussion Question 6 – Under-Escalation and Over-Escalation I agree that both under-escalation and over-escalation may result in harm and that the two types of error may not be directly commensurable. The potential severity of harms resulting from these different error modes may therefore influence Criticality Stratification. However, detailed weighting of the associated risks will often only be possible at the level of Product-Specific Risk Management, where product-specific probabilities and the effectiveness of implemented risk controls can be considered. For Criticality Stratification, only factors that can already be reliably established or bounded at this stage should be taken into account. This may include safeguards, such as defined escalation procedures integrated into the GenAI-enabled device, that can reliably constrain the consequences of over-escalation or under-escalation. I would also be cautious about treating escalation primarily as a binary problem of “escalate” versus “do not escalate.” GenAI-enabled devices may provide a spectrum of recommendations, for example: • continue self-management; • arrange routine medical review; • contact a healthcare professional within a defined period; • seek urgent care; • seek immediate emergency care. Errors may therefore involve not only false-positive or false-negative escalation, but also incorrect escalation level, inappropriate timing, or inappropriate destination of care. The Criticality Stratification should reflect these graded outcomes and the clinical consequences of moving a patient in either direction along this escalation spectrum. Section V – A Competency-Based Approach for Premarket Evaluation of GenAI-Enabled Devices General Considerations for Section V – including Responses to Discussion Questions 7–8 I generally support the competency-based approach proposed in the Discussion Paper, in particular the combination of device benchmarking and clinical confirmation and the concept of tailoring the nature and rigor of evidence to the device's intended use and criticality. Given the open-ended input and output space of GenAI- enabled devices, exhaustive testing is neither feasible nor an appropriate regulatory objective. However, I again suggest distinguishing these aspects more clearly. In particular, this applies to the already introduced delineation between Criticality Stratification and Product-Specific Risk Management. On the one hand, it should be demonstrated that the boundaries defined during Criticality Stratification are robustly established. Assumptions or safeguards that materially reduce the assigned criticality or evidence burden should themselves be supported by evidence. Thus, appropriate evaluation should be included for this. For example, where lower criticality is assumed because a qualified healthcare professional independently reviews an output, evaluation should establish not merely that a human approval step exists. It should validate that the intended users can realistically identify and correct safety-relevant errors under the intended conditions of use. Findings from evalua- tion should be capable of feeding back into the initial criticality assessment if such assumptions prove unreliable. Conversely, Criticality Stratification can serve as a basis for defining the required extent of premarket evaluation. In principle, this can be aligned with the two-axis framework described in the Discussion Paper. However, I would include additional aspects as discussed in the responses to Questions 1–6 in Section IV. Based on this, most evaluation activities will then be directed toward product development and, in particular, Product-Specific Risk Management. Ultimately, evaluation should provide reasonable assurance that the GenAI-enabled device is safe and effective. In this regard, different levels of criticality could be used to shape the evaluation requirements in a proportionate manner. This may support a staged evaluation approach in which criticality is not treated as completely static but can be reassessed step by step as evidence develops – see further comments below. The analogy to competency assessment of human clinicians is useful as an organizing principle, particularly with respect to structured assessment, supervised practice, progressively greater independence, and periodic reevaluation. The Discussion Paper itself describes the analogy as one that requires adaptation to the technical, practical, and legal characteristics of medical devices. However, I would be cautious to overemphasize the analogy to an assumption of equivalence between clinician and GenAI competency. Clinical professionals operate within an education, accountability, and professional-practice framework that differs materially from the mechanisms underlying GenAI systems. In addition, increasing reliance on AI may alter the capabilities and behavior of the human users themselves. For example, deskilling is an additional risk that should be considered with extensive use of GenAI in clinical settings. Human oversight should therefore be treated as a risk control whose effectiveness may require both initial and periodic evaluation, rather than as an inherently reliable safeguard. Distinction between errors and consequences Additionally, I suggest more clearly distinguishing between errors in the output of the GenAI system and the consequences of the errors in clinical practice. In scientific publications, the evaluation of AI systems often focuses on errors (e.g., accuracy rates) but not on the consequences of the different types of errors (e.g., false positives or false negatives). The consequences may strongly depend on the risk management measures and other safeguards implemented in the GenAI-enabled device or in the healthcare organization or clinical practice. On the one hand, benchmark tests often focus more on detecting errors in the output of the GenAI device. They may be weighted according to clinical impact where that impact can be assessed in a sufficiently generic way. Thus, they may also include an analysis of more general consequences. On the other hand, clinical confirmation is more directly oriented toward the outcome in a concrete clinical setting. Especially for GenAI devices whose performance usually depends on a wide range of parameters, e.g., use environments, clinical protocols, conversational paths, I agree that clinical confirmation is an important step that should be established separately. Although both evaluation parameters (errors and consequences) can be addressed in both steps (device benchmarking and clinical confirmation), I suggest clarifying these relationships. From my perspective, it is crucial to clearly identify which parameters are best addressed in each phase. Progressive evidence-based deployment The concept of "supervised practice with progressively greater independence" could be extended and operationalized more directly for GenAI-enabled devices. A device could initially operate within a narrowly defined and highly controlled operating envelope – for example, limited indications, specialist users, mandatory independent review, or restrictions on action-taking – and subsequently expand its scope or independence only when prespecified evidence thresholds have been met. This would also lead to a staged approach for stepwise expansion, e.g., through the following sequence: → strongly controlled conditions (e.g., restricted to dedicated experts or strong safeguards) → supervised clinical use → expanded use This would strengthen the analogy to HCP training, in which competence is established progressively through different levels of supervision and responsibility. Such an approach could use predetermined evidence gates, including prespecified performance and safety criteria, together with criteria for restriction or rollback if performance deteriorates. Where legally and practically appropriate, Predetermined Change Control Plan (PCCP)-like mechanisms may provide a useful model for prespecifying some such transitions. This could create a more direct link between premarket evaluation, supervised clinical deployment, periodic evaluation, and lifecycle change control.

Question 6: Under-escalation versus over-escalation. I encourage CDRH to allow explicit decision-analytic weighting rather than a single blended accuracy metric. Manufacturers would prespecify, per clinical context, the relative disutility of missed versus unnecessary escalation, justified with clinical evidence and, where available, health-economic estimates of downstream consequences; this is analogous to how screening programs weigh false negatives against false positives. Both error rates should be reported separately with confidence intervals, and acceptable operating points should be justified per indication, since a trade-off acceptable for pediatric sore throat triage is not acceptable for chest pain. Section V: Competency-Based Approach for Premarket Evaluation

Question 6 - Under-escalation and over-escalation Under-escalation and over-escalation should be measured as distinct error classes because their harms are different and often asymmetric. Under-escalation may create delayed diagnosis or treatment; over-escalation may create unnecessary emergency utilization, testing, anxiety, procedural risk, and eventual erosion of trust. Sponsors should prespecify clinically justified thresholds and report both directions of error rather than collapsing them into a single accuracy statistic. Threshold selection should reflect clinical context, the consequence of delayed care, the burden of unnecessary escalation, and the intended user population. SECTION V - COMPETENCY-BASED PREMARKET EVALUATION

Question 6 — Weighing under-escalation against over-escalation Don't just minimize errors — price them. Over-escalation drives unnecessary utilization, cost, and anxiety; under-escalation drives delayed diagnosis and preventable harm. These costs are neither symmetric nor universal across use cases. Rather than optimizing a single blended metric, FDA should require sponsors to define context-specific error costs before approval, and hold them to optimizing both directions of error for the intended use. Part II — Competency-Based Premarket Evaluation

Question 6 — How should FDA weigh under-escalation against over-escalation? Response Both are safety problems. A useful way to conceptualize triage risk is: Probability × Severity × Consequence of Delay The most statistically likely explanation is not always the safest basis for action. A low-frequency condition with catastrophic consequences if missed may appropriately warrant escalation even when a benign explanation is more probable. FDA should therefore evaluate whether AI recognizes low- frequency, high-consequence possibilities and weighs the harm of unnecessary escalation against the harm of failing to escalate. Under-escalation can delay necessary treatment and result in catastrophic harm. Over-escalation can unnecessarily direct patients to emergency departments, produce avoidable testing, increase cost, consume scarce clinical resources, create fear, and eventually cause patients to disregard warnings from systems that repeatedly overreact. The solution cannot be to allow AI systems to improvise urgency and level-of-care determinations from the unrestricted contents of the internet. That would be an extraordinary step backward. Healthcare already possesses evidence-based frameworks for determining clinically appropriate levels of care. MCG is one example. Its evidence-based criteria extend across the continuum of care and can inform questions such as whether a patient can safely receive care in an ambulatory environment, whether hospital care is necessary, and what intensity or level of hospital care is clinically appropriate—including medical- surgical, telemetry, intensive-care, and neonatal levels of care. That distinction is important. Level of care is a clinical safety determination, not simply an insurance-status determination. Telephone and telehealth triage provide another mature example. Schmitt-Thompson Clinical Content has spent more than 30 years developing rigorously reviewed adult and pediatric nurse-triage guidelines. Its published materials describe expert-panel review, annual updating based on changes in the medical literature and quality/outcome information, and disposition logic ranging from emergency intervention to self-care at home. [9][10] Deborah “Nurse Deb” Ault | Response to FDA Discussion Paper | Page 7 FDA-2026-N-7874 | Generative AI-Enabled Medical Devices A separate, complementary example is Wolters Kluwer/Lippincott’s Telephone Triage Protocols for Nurses, now in its seventh edition (2026), which uses systematic telephone-triage protocols to direct callers toward emergency care, clinician evaluation, or home-care instructions as appropriate. [11] Again, the point is not that FDA should mandate any particular proprietary guideline set. The point is: We already know how to put an evidence base behind level-of-care decisions. AI should build upon that body of work. It should not rediscover triage by reading the internet. The internet contains peer-reviewed research and excellent clinical guidance. It also contains outdated medicine, advertising, anecdotes, commercial influence, conspiracy theories, miracle cures, misinformation, and outright quackery. Allowing an AI to independently derive safety-critical level-of-care decisions from an undifferentiated universe of information would be indefensible when curated, continuously maintained clinical evidence already exists. For safety-critical functions such as triage, escalation, medical necessity, and appropriate setting and intensity of care, FDA should favor systems demonstrably grounded in curated, current, evidence- based clinical sources with identifiable provenance. AI may synthesize that evidence. AI may operationalize it. AI may explain it. AI may personalize its presentation. AI may help clinicians apply it more consistently. AI should not invent the clinical standard. The objective is neither maximal escalation nor minimal escalation. The objective is: the right patient receiving the right care at the right time in the right place. III. Competency-Based Evaluation

Question 6 — Under- and over-escalation The paper correctly observes that these two error directions may not be commensurable. My recommendation is procedural: rather than asking manufacturers to reconcile them, require that the trade-off be prespecified rather than discovered. A sponsor should state, before evaluation, an explicit asymmetric acceptance criterion — for example, a minimum sensitivity for the escalation-warranted condition together with a maximum acceptable over- escalation rate — accompanied by a clinical justification for the chosen ratio that references the specific deployment context and the capacity of the receiving system. The justification, not just the numbers, should be part of the submission and should be assessed. Post hoc reporting of both rates without a prespecified trade-off allows the sponsor to characterize whichever result is more favorable as the primary endpoint, and CDRH should foreclose that. Over-escalation deserves more attention than it typically receives. Its harms — alert fatigue, resource diversion, downstream harm to patients who did need the capacity consumed — are diffuse, delayed, and rarely attributed to the device. They are therefore systematically under-detected by exactly the postmarket mechanisms Section VI describes. 4 of 19 Docket No. FDA-2026-N-7874 II. Premarket evaluation and the competency-based approach (Section V; Questions 7– 17)

Question 6. Escalation error in both directions, and a third routing option We agree both under-escalation and over-escalation matter. We suggest the framework also recognize a third response beyond escalate and do not escalate. In our setting, a concerning finding is routed to the member's health plan care team, which holds the member's clinical history, the treating physician's contact information, and the authority to arrange follow-up. The output is not a direction to the member to seek emergency care, and it is not silence. It is a handoff to an accountable clinical channel carrying a defined response obligation. This third path changes the error profile substantially. The cost of a false positive is a care team review rather than an emergency department visit. The cost of a false negative remains serious and is the error we weight most heavily. We suggest CDRH consider routing to an accountable channel as a distinct activity category, because collapsing it into patient-facing escalation overstates the risk of an architecture designed specifically to reduce that risk. © 2026 Cara AI, Inc. On weighting the two directions of error, we do not think a single ratio generalizes across clinical contexts. What generalizes is the requirement for a sponsor to state the ratio it optimized for, justify it against the clinical context and the receiving system's response capacity, and report both error rates separately rather than inside a combined accuracy figure.

Question 6: escalation error in both directions. I see both failure modes in the emergency department every week. Under-escalation injures the patient in front of the device. Over-escalation injures the same patient through cascade testing, cost, and anxiety, and injures everyone else through crowding and triage dilution; over time it erodes the trust that appropriate care-seeking depends on. The two directions are real, asymmetric, and not commensurable, and they should not be collapsed into a single utility score. Sponsors should prespecify asymmetric error tolerances justified by clinical context and report both rates against clinician performance on the same scenario set. For DGBI functions, alarm features (gastrointestinal bleeding, unintended weight loss, progressive dysphagia, nocturnal symptoms) warrant near-zero tolerance for under-escalation, while a bounded, disclosed tolerance for conservative over-referral is acceptable and honest.

FDA Question 6 - Under- vs over-escalation Trace ID. TR-Q06 | FDA Q6; Sec. IV.A; App. B; pp. 9-10 / 27-28 BCR response. Maintain separate under- and over-escalation residuals with severity and timing. Do not force them into one score unless a clinical weighting rule is justified in advance; severe missed escalation can be a hard gate. BCR rule basis. BCR-R04,R05,R14,R16 Solution-stack link. S1,S4,S6,S7 Closure evidence. Separate under- and over-escalation rates with severity/timing; hard-gate misses tracked separately Pass / re-open. Each direction meets prespecified limits; catastrophic misses satisfy hard gate Re-open when: Clinical context, escalation policy, or threshold change.

Response to Question 6: Care Escalation — Asymmetric Harm Standards FDA's attention to care escalation is well-placed. Under-escalation — failure to recognize that a patient's condition warrants urgent intervention — represents the more severe harm in most clinical contexts, with potential for direct patient injury or death. Over-escalation — directing patients toward emergency care unnecessarily — creates system burden, patient anxiety, and economic costs, but is rarely life-threatening. FDA should establish asymmetric harm standards for escalation decisions: under-escalation threshold failures should be weighted more heavily in risk classification and performance evaluation. However, over-escalation cannot be dismissed entirely, particularly in resource-constrained environments where unnecessary escalations may deprive other patients of timely care. IV. SECTION V: PREMARKET EVALUATION — RESPONSES TO DISCUSSION QUESTIONS 7–17