Fault Line AI (Ben Carrington)

Please see the attached document for Fault Line’s full comment. We address Questions 1, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 19, 20, 22, 23 and 24 of the discussion paper, drawing on our research into clinical AI risk taxonomies and benchmark validity. Our comment covers the competency-based evaluation approach, benchmarking and clinical confirmation, the role of structurally independent evaluators, postmarket monitoring and change control and the two-axis risk framework.

Ben Carrington
Co-CEO, Fault Line AI
Submitted on behalf of Fault Line AI

FDA-2026-N-7874: Considerations for the Regulation of Generative AI-Enabled
Medical Devices: Discussion Paper and Request for Feedback
Dear Dr. Tarver and Dr. Abramson,

Fault Line welcomes the opportunity to comment on the discussion paper from the Center for Devices and
Radiological Health (CDRH) and its Digital Health Center of Excellence. We commend CDRH for seeking
input ahead of draft guidance and for engaging directly with the difficult questions in generative AI (genAI)
regulation.

For context, Fault Line is a clinical AI safety research organization that independently evaluates genAI
healthcare products. Our comments draw on that work and on our team’s previous experience taking
AI-enabled medical devices through regulatory authorization and scaling mental health clinical AI products
across the US and UK. In the interest of transparency, Fault Line provides independent research evaluation
services of the kind recommended in this letter.

Summary of position:
01 Automated evaluation is the best device benchmarking method for predicting real-world
performance

02 The evaluation system used to perform device benchmarking should be validated

03 Device benchmarking should be mandated across premarket evaluation and postmarket
monitoring

04 Benchmarking across the device lifecycle should be performed by structurally
independent organizations

05 Evidence from device benchmarking can be sufficient for market authorization

06 Benchmarks should be rerun through postmarket monitoring to create real-time
evaluations of ongoing performance and safety

Fault Line AI | Page 1
1. Benchmarking and Premarket Evaluation
Key Point #1: Automated evaluation is the best method for predicting the real-world
safety and performance of genAI medical devices

Key Point #2: Benchmarking should be performed pre- and postmarket

Key Point #3: Benchmarking taxonomies and evaluation approaches must be empirically
validated across multiple parameters

Question 7: The competency-based approach
We support the competency-based structure and consider the Appendix A schema broadly well
constructed. Device benchmarking should be used to:

■​ Establish the technical performance of a device against its intended use, and
■​ Quantify its risk profile and the technical performance that underpins its expected benefit

Human credentialing is a poor analogy. In clinical exams, performance is a reasonable proxy for
competence on related tasks, so clinicians are not tested on every scenario they may encounter. For
example, a doctor who manages a presentation correctly in an exam vignette can usually manage it when
the history arrives in a different order or in a patient's own words. GenAI does not share this property. Its
capabilities are jagged1, so strong performance on one presentation gives limited assurance about a slightly
different one. This means that testing on a wider range of scenarios that representatively cover the
real-world distributions is critical to ensure performance in real-world deployment is generalizable.
Consider licensing-exam benchmarks like MedQA, derived from the USMLE. Their questions are publicly
available, and because frontier models often do not disclose training data, the benchmarks themselves may
have been used in training. Further, emerging research has shown models may be prone to “shortcut
learning,” where models use cues based on question formats rather than generalizable clinical knowledge.
For instance, when MedQA was altered with fictional questions and modified with missing answers, models
performed worse. Separately, frontier models score highly on medical imaging benchmarks, even when no
image is provided.

CDRH should not accept performance on licensing-style exams as evidence of clinical proficiency. The
sponsor should carry the burden of showing its test cases are diverse enough to prevent degradation in
real-world use and empirically quantify claims about sufficient coverage of its synthetic testing data.

Automated evaluation should be the primary mechanism for premarket device benchmarking:

■​ It easily scales to the volume of interactions required to confidently validate genAI outputs
■​ It can test each risk from many different angles, adjusting mid-conversation, until it has covered the
realistic range of ways that risk can show up

Fault Line AI | Page 2
 Fig. 1 - Schematic representation of the components of automated evaluation

Human expertise remains important for defining the risk surface (i.e., the risk scenarios a product will
encounter given its intended use and user population), validating scoring rubrics, exploratory red-teaming
and sample-based review of live interactions in postmarket. However, human testing cannot cover enough
cases (in totality and in diversity) to give confidence that a genAI medical device is safe at scaled use.

Question 8: Evidence levels for competency-based approach
Whichever risk framework CDRH adopts, the evidence it requires should produce a statistically grounded
estimate of both the likelihood and the severity of each identified risk, reported by risk dimension (see
Question 12).

We argue throughout this letter that automated evaluation produces the evidence most predictive of
real-world safety and performance, provided the automated evaluation system is validated against specific
criteria (see below). As such, this could be a unified evidence baseline across devices. The size and scope of
the evaluation should then scale with the device’s risk categorization.

Questions 9 & 10: Benchmarking adequacy and validity
CDRH should specify requirements for device benchmarking in terms of:

■​ what is tested (the risk taxonomy)
■​ how it is tested (the evaluation methodology)

There are no standards yet for benchmark quality, so unguided use of benchmarks risks premarket
evidence being built on tests that were never validated for quantification of performance and safety
capabilities of genAI in clinical practice.

— The risk taxonomy
The taxonomy used needs to be devised from a methodological, structured (ideally data-driven) approach.
Currently this does not happen. Our research4 shows that existing clinical AI benchmarks:

1.​ Each tests a small slice of the risk space, and none covers all known healthcare AI risks (Fig. 2)
2.​ Lack realism and do not reflect how risks present in the real world (Fig. 3)

Fault Line AI | Page 3
Fig. 2 - Visual representation of risk coverage for established clinical AI benchmarks. No benchmark demonstrates having
at least 1 test case for every risk dimension. Adapted from Hou et al., 2026.

Fig. 3 - Real-world conversations (LMSYS, WildChat) have different failure mode correlation profiles and prevalence
compared to benchmark datasets. Adapted from Hou et al., 2026.

Fault Line AI | Page 4
A taxonomy used for device benchmarking should demonstrate:

01 Coverage and diversity ​
The taxonomy should span all foreseeable risk dimensions for the device’s intended use
(coverage) and the different ways each risk presents (diversity).

02 Quantified evidence of coverage​
This should be demonstrated with data to show the product risk surface has been adequately
covered by the benchmark.

03 Grounding in real-world data​
Risk dimensions and the test cases built from them need to be representative of real interactions
and risk presentations.

— The evaluation methodology
Almost no existing benchmark has itself been tested for whether it predicts real-world performance and
safety. A benchmark's core job is to separate safe from unsafe and performant from non-performant. Few
benchmarks are able to show this5. Current practice, which relies on static test cases and clinical vignettes
or personas, tends to measure average behavior on scenarios a device may have been tuned against. They
rarely stress-test the edges and bounds of a risk dimension, which is typically where serious failures occur.

Evaluation methodologies should be adversarial and adaptive. They should actively probe a product for
failure, adjust within an interaction to explore a diversity of outcomes and be run at a scale to be confident
the distribution of potential generative outputs has been sufficiently covered.

The only way to meet these criteria is through an automated evaluation approach that combines:

■​ Realistic simulation
Free-flowing conversation creates a combinatorial space that cannot be tested exhaustively with real
interactions, so simulated users are necessary. Simulation only predicts real-world performance when
simulated users behave like real ones. Simulations should therefore be seeded from and calibrated
against real interaction data. The evaluation system should be able to show that (1) simulated
conversations cannot be distinguished from real ones on measurable characteristics and (2) simulations
follow their assigned test scenarios instead of drifting into easier territory. This approach can also be
used to investigate rare, extreme risks (see Question 13).

■​ Multi-turn testing
We agree with CDRH that risk arises and compounds across conversational trajectories. Scope drift,
dependency and over-reassurance build up over many exchanges, and single-turn item sets cannot
detect them. The depth of multi-turn testing should be proportional to expected use. For example, a
device that summarizes a record update for a clinician needs fewer turns than a behavioral health
product delivering a CBT session.

Fault Line AI | Page 5
 ■​ Coverage and diversity
Coverage addresses whether the evaluation covers all the risks the device may encounter. Diversity
addresses whether the evaluation has tested all the ways a particular risk surfaces. These are separate
measures and so should be reported independently. For example, two evaluations can both test
suicidality (coverage) but one may explore hopelessness, indirect ideation, means restriction and erosion
of protective factors while the other runs twenty variants of self-harm disclosure (diversity).

— Validating the evaluator
If a device benchmark is to be used as premarket evidence for a device, it should be empirically validated
for its ability to predict how a clinical AI product will perform, and how safely, in real-world deployment.
This requires measurement on several properties including:

■​ Realism ■​ Within-dimension diversity
■​ Sensitivity ■​ Reliability
■​ Risk detection ■​ Adjudication validity
■​ Coverage

Analysis of leading benchmarks shows many cannot distinguish safe from unsafe genAI products and do
not realistically reflect real-world performance5.

Fig. 4 - Ability of leading clinical AI device benchmarks to distinguish safety-hardened (strong) from safety-unhardened
(weak) products. Adapted from Witte et al., 2026.

Fault Line AI | Page 6
Question 12: Statistically meaningful performance measurement
In addition to the above, the evaluation should document which characteristics and risks its synthetic test
cases were designed to cover. As discussed, synthetic data is especially valuable for rare, severe scenarios,
such as acute suicide risk, that real-world samples seldom capture.

To estimate risk, we recommend combining two sources: real-world data, which estimates how often each
risk scenario occurs in the intended-use population, and benchmark data, which estimates how often the
device fails when that scenario occurs. Together these give an empirical estimate of the probability that a
hazardous situation arises (P1 in ISO 14971 terms). The probability that the hazardous situation then leads
to harm (P2), and the severity of that harm, should be estimated per risk dimension. These estimates can
draw on the harm pathway (see Section 5) and the device’s position on the activity axis, since more
directive outputs are more likely to be acted on. Results should be reported by risk dimension with interval
estimates.

2. Clinical confirmation
Key Point #1: Automated evaluation is more representative of real-world safety
performance than clinical studies

Key Point #2: Evidence from benchmarking alone can be sufficient for market
authorization

Questions 11, 13 & 14: Clinical confirmation, synthetic data and comparators
Fault Line believes that synthetic testing, built on a scientifically grounded taxonomy and run with an
independently validated adversarial methodology, can determine a device's risk profile robustly enough to
support market authorization on its own. A well-powered, validated benchmark gives a more realistic view
of real-world risk than a prospective study. This is because:

Current clinical evidence practices for AIaMD are a weak baseline. Premarket studies are mostly
retrospective, often single-site and frequently rely on enriched datasets and settings that do not match the
intended use6,7. Many authorized AI devices lack appropriate clinical data altogether8,9,10,11. These problems
are exacerbated for genAI given the inherent stochasticity of the technology, particularly with patient-facing
voice agents and chatbots where output spaces cannot be limited sufficiently. As such, clinical studies of
feasible size will be underpowered to show that a risk not observed in the study is absent from the real
population.

Synthetic evaluation is the better predictor of real-world performance. The practical question for clinical
confirmation is how best to estimate real-world safety before deployment. Synthetic evaluations can:

■​ run at a scale that meaningfully covers the output space of generative devices
■​ be repeated under the same protocol across devices, versions and model updates, which makes
results auditable (and a natural vehicle for postmarket monitoring)
■​ target the rare, high-severity risk boundaries that clinical studies cannot reach

There are caveats here. For example, some high-risk products will not suit this approach. Clinical data is
still extremely important for validating the overall performance of a device, for example its impact on
clinical outcomes. On balance, however, we do not believe the clinical confirmation activities listed in the

Fault Line AI | Page 7
discussion paper should necessarily be required for premarket evaluation if the approaches above are
followed.

3. Independent third parties
Key Point #1: Structurally independent organizations should perform pre- and postmarket
benchmarking activities

Key Point #2: There should be an accreditation framework for independent organizations,
drawing on existing third-party programs.

Question 16: Role of independent third parties
Independence is the trust and accountability layer this class of device needs if it is to improve patient
outcomes. The AI safety community is calling for independent auditing and evaluation of genAI
models12,13,14,15, yet clinical AI, where the stakes are arguably highest, is lagging behind. Independence
should be operational as well as institutional. For example, an independent evaluator should not share a
reference standard or synthetic data generator with the device it evaluates.

Independence delivers the following:

■​ Removes analytical freedom
Without independent benchmarks and clear quality standards, a sponsor can choose the benchmark that
shows what it wants to show, which makes internally driven testing perfunctory. This freedom is
comparable to the analytical freedom that made pre-registration of endpoints and analyses a standard
requirement for RCTs.

■​ Accelerates market access for sponsors
An ecosystem of third-party standards built to FDA specification means sponsors no longer need to
invest heavily in building and maintaining bespoke testing that has no certainty of meeting FDA
expectations.

■​ Produces safer, more performant products
Third-party evaluation exists in other high-stakes fields (e.g., cybersecurity) because it is one of the most
effective ways to find failures before they cause harm. AI safety evaluation is also a young research field
with hard scientific problems. It is unrealistic to expect every clinical AI company to meet the required
safety standard, as the weakness of the current benchmark and testing landscape shows. As genAI
products move toward autonomously delivering care, the stakes for patients get higher. In this context, it
would be unethical not to involve third parties in the evaluations of these technologies and devices.

CDRH should implement an accreditation framework for organizations that evaluate genAI devices. There is
clear precedent for its dimensions, including credentials, compliance, evidence of platform validation,
transparency and impartiality16,17,18,19. FDA already runs comparable programs in the 510(k) Third Party
Fault Line AI | Page 8
Review Program and the Medical Device Development Tools (MDDT) Program from which inspiration can be
drawn.

Independent evaluation of AI is already being written into law in the United States and elsewhere, including
for healthcare and consumer-facing products16,20,21,22. CDRH has the opportunity to hold genAI devices to at
least the same standard.

4. Postmarket Monitoring
Key Point #1: Device benchmarks can establish safety and performance thresholds
measured in real-time postmarket

Key Point #2: Postmarket reporting cadence should be increased in line with this approach

Key Point #3: Change control should be anchored to predetermined thresholds, instead of
specified changes.

Question 18: Greater reliance on postmarket monitoring
We agree with shifting the balance of evidence demands from premarket to postmarket. Premarket
benchmarking establishes a device’s baseline safety and performance profile and postmarket monitoring
confirms that this profile holds on the real-world distribution of users and interactions. This continuity is
essential for genAI devices. However broad the premarket evaluation, the input and output space cannot be
tested exhaustively before launch, as it largely can for conventional SaMD or deterministic AIaMD.
Postmarket surveillance should therefore carry more of the evidence burden and be performed and
reported on a more frequent cadence than is typical today.

The exact cadence can be agreed per device based on intended use, risk, data availability and the type of
activity. For example, whether an AI therapy device reliably follows its crisis protocol is easy to measure
and central to its safety, so it might be reported weekly. Identifying downstream adverse events such as
self-harm requires data to pass between several stakeholders, so semiannual reporting may be the realistic
limit. Especially for examples like the latter, the burden of collecting and reporting postmarket data cannot
rest on the sponsor alone, which often has little influence on the collection of required data.

Fault Line AI | Page 9
Questions 19 & 20: Postmarket evaluation approaches

Fig. 5 - A total lifecycle approach to evaluation of genAI medical devices where the premarket evaluation baseline is
rescored in real-time on postmarket live data to track performance, improve evaluation simulation and enable
performance-based, rather than item-based, predetermined change control.

The premarket benchmark’s graders and thresholds should be applied to real-world data throughout the
postmarket phase to measure change against established safety and performance metrics and the device's
overall risk profile. Here, automated graders are the best option, given (a) they can run over a larger volume
of real-world data to review every interaction instead of sampling, (b) on validated dimensions, they match
or exceed inter-rater agreement among clinical experts and (c) they can apply criteria consistently2,3,23,24.

These automated supervisory agents and graders can run at the device level and assess subgroup
performance against the intended use, at scale and near-continuously. Paired with a rerun of the premarket
benchmark before each update ships, this plays the role that regression testing plays in conventional
software: degradation from an update is caught before release and anything the benchmark missed is
detected in live use far sooner than periodic review would allow. This system can be set up in a way that
minimizes data retention, preserving the security posture of the device. For example, graders can score
interactions as they occur and keep only the scores, flags and the minimum record needed to reconstruct a
flagged case without retaining every transcript.

Fault Line AI | Page 10
Questions 22, 23 & 24: Change control, PCCPs and foundation model changes
The same methodology used for postmarket monitoring also supports device change control. As recently
noted by the MHRA’s National Commission into the Regulation of AI in Healthcare25, continuous monitoring
is a better basis for PCCPs. The current PCCP model, built around enumerating specific planned
modifications, is a poor fit for genAI devices. The most consequential changes, for example foundation
model updates, are often outside the sponsor’s control and impossible to specify in advance.

We believe there is an opportunity for CDRH to rethink the PCCP for genAI so that the modification protocol
is anchored in performance: a validated benchmark baseline, prespecified safety and performance
thresholds, and continuous monitoring against them. Any change that keeps the device within those
thresholds, whatever its source, should be deployable without a new marketing submission, and any breach
should trigger rollback and reporting.

5. Device Risk Assessments (Section IV)
Key Point #1: There are multiple harm pathways for genAI that should be defined by
sponsors

Key Point #2: The probability of genAI risks occurring can be quantified in line with
existing medical device risk management frameworks

Question 1: Dimensions of the two-axis risk framework
Directionally, the two-axis framework makes sense. We recommend aligning this framework with existing
risk estimation activities, e.g., ISO 14971. The axes map well to the existing notions of severity
(consequence) and probability/likelihood (activity/directiveness), in the sense that the activity axis will be
strongly correlated with the likelihood of an AI action leading to harm.

We suggest CDRH:

Broaden the consequence axis. The paper frames consequence as the result of "relying on an incorrect
output." That is only one route to harm among several. Some harms occur on exposure to the output itself.
Others build up across a conversation or repeated use, where each individual response may be factually
correct.

Classify hazards by harm pathway. We suggest sponsors classify each genAI hazard by the route through
which it causes harm. Distinguishing harm types matters because it forces sponsors to critically assess the
appropriateness of associated mitigations. For example, the mitigations in the discussion paper (e.g.,
clinician review, disclaimers) mainly act on the “action” pathway but do little for exposure or cumulative
harms. Likewise, whether a user can evaluate an output is relevant to action harms and has little bearing on
the others. Cumulative harms are also why multi-turn evaluation is essential (see Section 1).

Harm pathway types Example

Exposure: harm on reading or hearing the The device responds to an emotional disclosure in a way
output that invalidates or blames the user

Action: harm when the user acts on the The device recommends a drug that is contraindicated or
output interacts with a current medication

Fault Line AI | Page 11
Harm pathway types Example

Omission: harm from what the device The device under-triages an urgent presentation and never
failed to say or ask tells the person to go to the ER

Cumulative: harm that builds over time The device becomes the user's main source of support and
encourages withdrawal from other people

Require evidence that clinician review works. Clinician supervision should count as a risk control only
where there is evidence it works under realistic time pressure, workload and automation bias. For example,
if a reviewer cannot practically detect an error produced by an AI summarization, the act of the review
cannot be an effective mitigation for summarization error rates.

Question 6: Under-escalation and over-escalation
We agree both should be evaluated. They should be measured and reported separately, each against a
prespecified and clinically justified threshold. Under- and over-escalation should not be combined into a
single accuracy figure.

We thank CDRH for the opportunity to comment and would be glad to discuss any of these
recommendations further.

Ben Carrington​
Co-CEO​
Fault Line AI

Fault Line AI | Page 12
6. References
1.​ Dell’Acqua, F. et al. (2023) ‘Navigating the jagged technological frontier: field experimental
evidence of the effects of AI on knowledge worker productivity and quality’, Harvard Business
School Technology & Operations Mgt. Unit Working Paper, No. 24-013. Available at:
https://doi.org/10.2139/ssrn.4573321

2.​ Arora, R.K. et al. (2025) ‘HealthBench: evaluating large language models towards improved human
health’, arXiv preprint, arXiv:2505.08775. Available at: https://arxiv.org/abs/2505.08775
(Accessed: October 11, 2026).

3.​ Bedi, S. et al. (2025) ‘MedHELM: holistic evaluation of large language models for medical tasks’,
arXiv preprint, arXiv:2505.23802. Available at: https://arxiv.org/abs/2505.23802 (Accessed:
October 11, 2026).

4.​ Hou, K. Y., McFadyen, J., Ereira, S., Witte, K., Nour, M. M., Carrington, B., Prichard, G., & Rollwage, M.
(2026). Towards a data-driven taxonomy of clinical AI risks. Zenodo.
https://doi.org/10.5281/zenodo.23350741

5.​ Witte, K., Hou, K. Y., Ereira, S., Carrington, B., Prichard, G., & Rollwage, M. (2026). Who Audits The
Auditors – A Framework For Evaluating Clinical AI Evaluations.

6.​ Muralidharan, V. et al. (2024) ‘A scoping review of reporting gaps in FDA-approved AI medical
devices’, npj Digital Medicine, 7, 273. Available at: https://doi.org/10.1038/s41746-024-01270-x

7.​ Chouffani El Fassi, S. et al. (2024) ‘Not all AI health tools with regulatory authorization are clinically
validated’, Nature Medicine, 30(10), pp. 2718–2720. Available at:
https://doi.org/10.1038/s41591-024-03203-3

8.​ van Leeuwen, K.G. et al. (2021) ‘Artificial intelligence in radiology: 100 commercially available
products and their scientific evidence’, European Radiology, 31(6), pp. 3797–3804. Available at:
https://doi.org/10.1007/s00330-021-07892-z

9.​ Antonissen, N. et al. (2026) ‘Artificial intelligence in radiology: 173 commercially available products
and their scientific evidence’, European Radiology, 36(1), pp. 526–536. Available at:
https://doi.org/10.1007/s00330-025-11830-8

10.​Windecker, D. et al. (2025) ‘Generalizability of FDA-approved AI-enabled medical devices for
clinical use’, JAMA Network Open, 8(4), e258052. Available at:
https://doi.org/10.1001/jamanetworkopen.2025.8052

11.​Abulibdeh, R. et al. (2026) ‘1,357 AI medical devices cleared, 3 actually tested on patient
outcomes’, PLOS Digital Health, 5(8), e0001597. Available at:
https://doi.org/10.1371/journal.pdig.0001597

12.​AI Evaluator Forum (2026) Minimum conditions for embedding evaluators [Open letter]. Available
at: https://aievaluatorforum.org/initiatives/embedded-evaluation-letter (Accessed: October 11,
2026).

13.​Brundage, M. et al. (2026) ‘Frontier AI auditing: toward rigorous third-party assessment of safety
and security practices at leading AI companies’, arXiv preprint, arXiv:2601.11699. Available at:
https://arxiv.org/abs/2601.11699 (Accessed: October 11, 2026).

Fault Line AI | Page 13
14.​Longpre, S. et al. (2025) ‘Position: in-house evaluation is not enough. Towards robust third-party
evaluation and flaw disclosure for general-purpose AI’, Proceedings of the 42nd International
Conference on Machine Learning, PMLR 267, pp. 81728–81758. Available at:
https://proceedings.mlr.press/v267/longpre25a.html (Accessed: October 11, 2026).

15.​Raji, I.D. et al. (2022) ‘Outsider oversight: designing a third party audit ecosystem for AI
governance’, in Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AIES
‘22). New York: ACM, pp. 557–571. Available at: https://doi.org/10.1145/3514094.3534181

16.​European Parliament and Council of the European Union (2024) Regulation (EU) 2024/1689
(Artificial Intelligence Act), Article 31: Requirements relating to notified bodies. Official Journal of
the European Union, L, July 12, 2024. Available at: https://www.euaiact.com/article/31 (Accessed:
October 11, 2026).

17.​California State Legislature (2026) Assembly Bill No. 1405: Artificial intelligence: auditors:
registration. Chapter 178, Statutes of 2026. Available at:
https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=202520260AB1405 (Accessed:
October 11, 2026).

18.​The Joint Commission and Coalition for Health AI (2025) Guidance on the responsible use of AI in
healthcare (RUAIH). Oakbrook Terrace, IL: The Joint Commission. Available at:
https://digitalassets.jointcommission.org/api/public/content/dcfcf4f1a0cc45cdb526b3cb034c68c
2?v=3edb8a95 (Accessed: October 11, 2026).

19.​National Institute of Standards and Technology (2024) Artificial intelligence risk management
framework: generative artificial intelligence profile (NIST AI 600-1). Gaithersburg, MD: NIST.
Available at: https://doi.org/10.6028/NIST.AI.600-1

20.​California State Legislature (2026) Senate Bill No. 813: Independent verification organizations.
Chapter 179, Statutes of 2026. Available at:
https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=202520260SB813 (Accessed:
October 11, 2026).

21.​Illinois General Assembly (2026) Senate Bill 315: Artificial Intelligence Safety Measures Act. 104th
General Assembly. Available at:
https://www.ilga.gov/Legislation/BillStatus/FullText?GAID=18&DocNum=315&DocTypeID=SB&Leg
Id=0&SessionID=114 (Accessed: October 11, 2026).

22.​California State Legislature (2026) Senate Bill No. 1119: Companion chatbots: children’s safety.
Chapter 190, Statutes of 2026. Available at:
https://calmatters.digitaldemocracy.org/bills/ca_202520260sb1119 (Accessed: October 11,
2026).

23.​Rollwage, M. et al. (2026) ‘A cognitive layer architecture to support large-language model
performance in psychotherapy interactions’, Nature Medicine, 32(5), pp. 1717–1725. Available at:
https://doi.org/10.1038/s41591-026-04278-w

24.​Li, C. et al. (2026) ‘A scoping review of LLM-as-a-judge in healthcare and the MedJUDGE
framework’, arXiv preprint, arXiv:2604.25933. Available at: https://arxiv.org/abs/2604.25933
(Accessed: October 11, 2026).

25.​Medicines and Healthcare products Regulatory Agency (2026) National Commission into the
Regulation of AI in Healthcare: recommendations for a future regulatory framework. London:

Fault Line AI | Page 14
MHRA. Available at:
https://assets.publishing.service.gov.uk/media/6aa2662962ec7fe7bedf52b9/Full_National_Comm
ission_Report_-_Publication__1_.pdf (Accessed: October 11, 2026).

Fault Line AI | Page 15