Would voluntary Foundation Model Master Files be practical, and useful in premarket review?

The question as posed CDRH asks whether voluntary Foundation Model Master Files would be practical for foundation model developers to provide and to keep current, and what content they might include. It asks what would make such a program sufficiently useful for premarket review, given that participation would be voluntary and that model developers may have limited incentive to disclose safety-relevant information. And it asks whether there are alternative mechanisms CDRH should consider for obtaining information about underlying foundation models. Summary of position We support the master file concept and we do not think it can carry the weight the question hopes for. Our comment has four parts, and the first is a measurement rather than an opinion. • The binding constraint is currency, not candour. The question anticipates that developers may be unwilling to disclose. We measured something more basic: on one ordinary workday, the model answering a widely used hosted assistant was exchanged 153 times in fourteen hours — a median of 41 seconds at a time. A document cannot be kept current against a quantity that changes every few minutes. A fully cooperative, entirely honest developer filing in good faith would produce a master file that was stale before it was read. • A second, unrelated provider showed the same class of behavior on the same machine on the same day, in a different rhythm. This is a property of hosted delivery, not the practice of one company, and a master file program has to be designed for it rather than around it. • The most important field in any master file is the one most likely to be omitted: where the model executes. "The model runs on the device" and "the device reaches a model" are different regulatory Docket FDA-2026-N-7874 · The Christman AI Project and Robotics Division Page 1 situations with different evidence behind them. We measured a model registered inside a local runtime, reachable at the loopback address of our own machine, that was in fact answering from a datacenter. • There is an alternative mechanism and it does not require the developer's participation at all. A record produced on the user's side of the connection can establish which model answered and where it ran, timestamped, append-only, with no credentials and no cooperation. We have published such an instrument under a permissive license rather than proposing one. We also recommend a second mechanism that puts the burden on the device rather than on a registry. 1. What we measured, and why it decides the question On 2026-09-08 an observation tool running on a single desktop workstation kept an append-only ledger of which models were answering on that machine. The figures below are computed directly from that file. All times are UTC. 1.1 The first hosted assistant Across fourteen hours and sixteen minutes — 04:59:30 to 19:15:39 — the ledger recorded 307 model-layer events: 154 substitutions and 153 displacements. These resolve into 153 complete cycles in which the assistant's named live model was displaced by a chat model operating from a remote datacenter and then reinstated. That is 10.7 substitutions per hour. The substituted model held the session for a median of 41 seconds, with the middle half of cycles between 33 seconds and 1 minute 6 seconds, and a full range of 25 seconds to 4 minutes 9 seconds. 141 of the 153 cycles lasted under two minutes. In total the substituted model held the session for 139.9 minutes — 16.3 percent of the observed day. The interval between cycles was not stable in the same way: a median of 1 minute 48 seconds, ranging from 8 seconds to 49 minutes 13 seconds. A narrow inner duration inside a highly variable outer one is the signature of a triggered process rather than a schedule. Nothing in the assistant's interface disclosed any of this. The operator states he did not open that assistant at any point during the window. That is his account, recorded as such and not independently verified by the instrument. 1.2 The second hosted assistant A second hosted assistant from an unrelated provider was resident on the same machine across nearly the same window. It produced 24 model-layer events and 10 complete cycles in a quite different shape: one unbroken hold of 7 hours 8 minutes, from 05:04:17 to 12:12:01, followed by nine cycles compressed into a 34-minute burst with a median hold of 2 minutes 29 seconds. After the burst the ledger records three further substitutions with no closing displacement before the window ended. Two providers, one machine, one day, the same class of undisclosed substitution at two different tempos. We name neither, because the behavior is the finding and naming a company converts a technical result into a dispute about a company. 1.3 What did not do this Docket FDA-2026-N-7874 · The Christman AI Project and Robotics Division Page 2 Five other named systems appear in the same ledger over the same window — editors, a local model runtime, an agent extension, a m

R3. Foundation model transparency should be mandatory, or the full evidence burden should fall on the sponsor (Question 25) This follows from S3 and S6. CDRH has already identified the problem with voluntary, confidential Foundation Model Master Files: developers "may have limited incentive to disclose safety-relevant information." A voluntary program will attract disclosure from developers who have little to hide and none from those who have the most. The result is a review process that is better informed about the safest models and least informed about the riskiest. I recommend a simple rule. If a sponsor builds a device function on a third-party foundation model, then either (a) the model developer's technical documentation — architecture, training data provenance, characterized limitations and failure modes, safety-relevant behavioral constraints, update policy, and audit log availability — is made available to FDA as a condition of the sponsor's submission, or (b) the sponsor treats the model as an unvalidated black box and carries the full evidence burden for every competency element without any reliance on the developer's claims. This aligns incentives: developers who want their models used in regulated devices will disclose, and sponsors who choose opaque models will pay for that choice in evidence rather than passing the cost to patients. The European Union's obligations on general-purpose AI providers — mandatory technical documentation and training-data summaries — are a workable reference point. I further recommend that the model card or system card for any foundation model used in a marketed device be public, not confidential. The Master File mechanism protects proprietary manufacturing detail, which is appropriate. But the characterized limitations and failure modes of a model that is generating clinical outputs are not trade secrets; they are labeling.

Response to Discussion Question 25 If voluntary Foundation Model MAFs proceed, I suggest the contemplated content include, alongside architecture and training provenance: refusal behavior, content-policy changes between versions, and output stability under varied user framing — including the speaker-authority framing described in Section 1 above. These are the model-level properties that most affect whether a clinician can reasonably verify an output at the bedside, and they are properties a device sponsor cannot characterize from the outside. A sponsor cannot evaluate what the MAF does not disclose. On the incentive problem the question raises: one practical lever is that a documented Foundation Model MAF would allow sponsors to satisfy portions of the re-examination described in Section 3 above by reference, rather than by independently re-characterizing the model after every upstream update. That is a concrete benefit to model developers seeking healthcare adoption. 5. On generalizability and deployment populations

Question 25: Foundation Model MAFs A Foundation Model MAF can provide useful dependency information. It cannot prove that a final device is safe or effective. Useful MAF content includes: The model and version identifier. The supported data types. The intended and unsupported uses. Known failure modes. Safety controls. Healthcare test results. Output variation. Change history. Update notice requirements. Incident information. Version control and rollback options. Newton’s Tree Inc Considerations for the Regulation of Generative AI-Enabled Medical Devices FDA Docket No. FDA-2026-N-7874 Service withdrawal plans. The manufacturer must still test the final device. A voluntary MAF can reduce repeated work. However, CDRH should require another method when the model provider does not give sufficient information.

3.9 Question 25 - Foundation Model Device Master Files The FDA considers whether voluntary Foundation Model Device Master Files could allow confidential information about underlying models to support downstream regulatory submissions (FDA, 2026a). Sitora supports the principle because downstream developers may retain responsibility for a regulated product while lacking complete information about the model on which that product depends. A useful master file could include a stable model and version identifier, architecture description, relevant data provenance, known limitations, healthcare-relevant failure modes, benchmark and subgroup results, safety controls, material update history, expected update cadence, deprecation policy and audit capabilities. The existence of such a file should not constitute general FDA approval of a model for medical use; responsibility for the specific device and intended use should remain with the downstream manufacturer.

Question 25 - Voluntary Foundation Model Device Master Files Voluntary foundation-model master files or analogous referenceable regulatory files [1,14] could be useful because they may reduce duplicated review and improve consistency when multiple device sponsors depend on the same underlying model. Their value will depend on whether the file contains sufficiently current, safety-relevant information and whether changes are communicated in a way that sponsors can operationalize. Useful content may include model identity and versioning, architecture and training-data provenance at an appropriate level of detail, intended supported use domains, known limitations and failure modes, healthcare-relevant evaluation results, subgroup performance, safety constraints and guardrails, tool-use characteristics, audit-log availability, update history, and change-notification commitments. A Foundation Model MAF should not substitute for sponsor-specific validation of the final deployed device. The same foundation model may behave differently depending on prompts, retrieval, tools, contextual memory, user interface, workflow, and local policies. The sponsor should remain responsible for demonstrating the safety and effectiveness of the configured device for its intended use. If voluntary participation is insufficient, alternative mechanisms could include standardized sponsor attestations, contractual disclosure requirements between model providers and device manufacturers, independent third-party assessments, machine-readable version metadata, and referenceable standardized model/system cards.

Question 25 — Foundation Model Master Files Voluntary disclosure won't be enough on its own for the highest-risk systems. A Foundation Model MAF covering training-data provenance, known limitations, safety-relevant behaviors, and update policies is a good concept — but foundation-model developers have limited incentive to volunteer their own weaknesses. Pair the voluntary MAF with contractual disclosure requirements for medical-device use, plus FDA authority to request more when necessary. A foundation model doesn't need FDA approval to exist — but once it materially shapes a regulated device's behavior, the information needed to evaluate that device cannot stay proprietary.

Question 25. A voluntary Foundation Model Master File can be useful if its contents are versioned, updateable, and contractually referenced. A Foundation Model MAF should include model identity and lineage; intended supported uses; architecture summary; training-data provenance at an appropriate level; healthcare-relevant evaluations; subgroup results; known limitations and failure modes; safety constraints; cybersecurity practices; update policy and notification commitments; deprecation and rollback terms; and available audit signals. FDA should permit tiered disclosure so proprietary details can remain protected while sponsors receive the safety information necessary to manage risk. The MAF should feed, but not replace, a sponsor-maintained AI-SBOM for the complete medical- device function. The AI-SBOM should supplement applicable existing design, product-BOM, hardware, software, supplier, and section 524B cybersecurity-SBOM records. It should identify the model and fine-tunes; prompts and configuration; retrieval sources and embedding components; tools, sensors, actuators, and external services; agents, memory, and orchestration; runtime and infrastructure dependencies; safety policies; identity and authority relationships; competency evidence; and applicable change lineage. Existing component records should be referenced by stable identifiers rather than copied. Protected supplier records may be referenced rather than publicly disclosed, provided the sponsor can obtain timely safety- relevant information.

Question 25 also asks what would make the program sufficiently useful given limited developer incentive to disclose. I note that a MAF describing behavior only in the developer's own reference configuration is both cheaper to produce and less probative, so the incentive gradient runs toward the version carrying the least evidentiary weight. Recommendations: • Where a submission references a Foundation Model MAF, the submission should document the transformations applied downstream, including fine-tuning, compression method and precision, and the deployed inference configuration, so the referenced evidence can be assessed for applicability. • MAF content guidance could ask developers to characterize the stability of safety-relevant behaviors under common downstream modifications, including quantization at standard precisions, rather than only in the reference configuration. 7. Adversarial benchmark exposure (Question 10)

Question 25 — Should FDA establish voluntary Foundation Model Device Master Files (MAFs)? Response FDA’s proposed voluntary Foundation Model Device Master File (MAF) concept could improve efficiency and reduce duplication if it allows FDA to evaluate important characteristics of an underlying foundation model once and permit downstream manufacturers to reference appropriate information in individual premarket submissions. [1] But voluntary transparency should not become a substitute for required safety information where downstream patient safety depends upon it. If a medical-device developer builds upon a third-party foundation model, the developer still remains responsible for demonstrating that its own product is safe and effective for its intended use. FDA should also be cautious about creating a framework in which downstream developers can say: “We relied upon the foundation-model developer’s file.” The existence of a Master File should not diffuse accountability. There should be clarity regarding:  what information the foundation-model developer is responsible for;  what the downstream manufacturer must independently validate;  how model changes are communicated;  when changes invalidate prior reliance;  and who is responsible when the interaction between the foundation model and the downstream application creates unexpected risk. The central principle should remain: Shared technical infrastructure does not eliminate product-level accountability. X. Questions Where We Do Not Offer Detailed Technical Comment

Question 25 — Voluntary Foundation Model Master Files The paper’s candor about the incentive problem is warranted. Foundation model developers serve markets in which medical devices are a small fraction of revenue; disclosure of safety-relevant limitations carries competitive and litigation exposure; and a voluntary program asking them to document weaknesses for a regulator overseeing someone else’s product is unlikely to attract broad participation on its own merits. I nonetheless recommend CDRH create it, because the mechanism that makes it work is not the model developer’s incentive — it is the device manufacturer’s. Pair the voluntary MAF with a manufacturer-side default: where an MAF is absent, the device manufacturer bears the full characterization burden for the underlying model. Training data provenance to the extent determinable, contamination assessment, known failure modes, version and deprecation 16 of 19 Docket No. FDA-2026-N-7874 practices, subgroup performance characterization — all of it, generated independently at the manufacturer’s cost, or the submission is incomplete. That default creates a procurement preference. Device manufacturers will strongly prefer model providers that maintain a current MAF, because the alternative is expensive and often technically impossible. Providers will then face a commercial reason to participate that has nothing to do with regulatory goodwill. This is how a voluntary program acquires teeth without new authority: it does not compel the model developer, it makes the device manufacturer want to compel them, and the device manufacturer is the party with the contract. Suggested MAF content: model identity and version lineage; training data cutoff date (essential for the contamination analysis under Question 10); versioning, change notification, and deprecation policy (essential for Question 24); known limitations and documented failure modes; safety training and refusal behavior; evaluation results the developer has generated; and any medical-domain-specific training or fine-tuning. A currency requirement — the MAF must be updated within a defined period of any model change — is what makes it useful rather than a snapshot. I would add one caution: an MAF should never be treated as establishing the safety or effectiveness of any device built on the model. The paper’s statement that the unit of evaluation is the final user-facing device as configured for deployment is correct and important, and the MAF should be framed as context for that evaluation, not as a partial substitute for it.

Point One: Voluntary Foundation Model Master Files (Question 25) The paper directly asks what would make a voluntary Foundation Model MAF program sufficiently useful given that model developers may have limited incentive to disclose safety relevant information. This is the right question to ask, and the paper's own framing already identifies the core weakness. Organizations with strong internal incentives to understand their own systems' behavior, including several leading AI developers, have this year experienced real world incidents in which anomalous or unsafe model behavior went undetected for extended periods, in some cases weeks or months, and was ultimately discovered through external review or by accident rather than through the organizations' own monitoring processes. These were not organizations lacking sophistication or motivation to monitor their systems. If developers with the strongest possible incentive to know what their own models are doing have repeatedly failed to detect meaningful deviations in their own systems, a voluntary model card or system card filed by that same party carries the same structural limitation: it can only disclose what the filer already knows, and this year's pattern indicates that what a developer believes it knows about its own system's behavior is not reliable. I recommend CDRH consider requiring, as a condition of a Foundation Model MAF being referenced in a premarket submission, that the filer disclose its own internal detection and monitoring methodology for the behaviors and failure modes described in the card, so that reviewers can evaluate not just the claimed behavior of the model but the reliability of the process that produced the claim. Point Two: Postmarket Monitoring Completeness (Questions 18 and 19) Question 18 asks what characteristics of a monitoring program would need to be in place to justify reduced premarket evidence. I recommend that any postmarket monitoring program accepted as a basis for reduced premarket evidence be required to demonstrate the completeness of its own coverage, not merely the presence of a monitoring process. A monitoring program that reports zero adverse events could mean the device performed safely, or it could mean the monitoring failed to observe the relevant behavior. These two states are not distinguishable from the absence of a report alone, yet a regulatory framework built around greater reliance on postmarket monitoring needs to distinguish between them. I recommend that sponsors be required to provide affirmative evidence of monitoring coverage, for example documentation of what proportion of real world interactions were captured by the monitoring system and what categories of device behavior fall outside its observable scope, alongside any adverse event reporting. Point Three: Independent Detection of Performance Degradation (Questions 19 and 24) The paper names performance degradation over time as one of the core risks of GenAI enabled devices, and proposes performance degradation monitoring as a postmarket approach. I recommend that CDRH require degradation detection mechanisms to operate independently of the deployed device's own output or reporting pathway. A device that is degrading, whether due to a change in the underlying foundation model, drift in the input population, or another cause, may not reliably signal its own decline through the same channel used to generate its outputs, particularly for GenAI enabled devices where a single failure mode can affect both the device's clinical output and its own self assessment of that output. This concern is closely related to Question 24, regarding changes initiated by a third party foundation model developer rather than the device manufacturer. In both cases, the manufacturer's ability to detect a problem depends on a signal that is generated or mediated by the same system that may be experiencing the problem. I recommend that degradation monitoring specifications require at least one detection mechanism, such as periodic re benchmarking against an independent reference set or sampled clinician review as described in Section VI.A, that does not depend on the device's own reporting of its performance. Conclusion The concerns raised above are not arguments against the competency based framework or against reliance on postmarket monitoring generally. Postmarket monitoring, done well, is likely necessary given the practical limits of premarket testing for open ended systems described elsewhere in this paper. My recommendation is narrower: that CDRH require monitoring and disclosure mechanisms to demonstrate their own reliability and completeness as a condition of being credited toward reduced premarket evidence, rather than treating the existence of a monitoring program or a voluntary disclosure mechanism as sufficient on its own. Given this year's demonstrated pattern of AI developers failing to detect problems in their own systems despite strong incentives to do s

Questions 24 and 25: third-party foundation models. A small manufacturer cannot compel a foundation model developer to disclose changes or give advance notice, and contractual leverage is concentrated in the largest sponsors. Four mechanisms would help. Treat model version pinning as a baseline design expectation, with developers disclosing deprecation timelines. Advance the voluntary Foundation Model MAF program with update notification commitments and healthcare-relevant evaluation summaries as core content, and signal that devices built on MAF-holding models will see more predictable review; that signal creates the developer's incentive to participate. Endorse sponsor-side re-benchmarking gates, under which no underlying model change enters production until the premarket benchmark battery has been re-run and passed, consistent with Section VI.C. And define a PCCP category for like-for-like model version upgrades validated through that prespecified re-benchmarking. Together these convert an uncontrollable third-party dependency into a controlled, auditable change process inside the sponsor's quality system.

FDA Question 25 - Voluntary Foundation Model MAFs Trace ID. TR-Q25 | FDA Q25; Sec. VII.C; App. B; pp. 23 / 30 BCR response. Foundation Model MAFs can help if current, versioned, and safety-relevant. Include architecture/interface boundaries, appropriate training-data provenance, supported uses, known limitations/failure modes, subgroup evaluations, guardrails, version history, update commitments, and audit-log availability. Voluntary gaps remain explicit and sponsors still need independent dependency assurance. BCR rule basis. BCR-R01,R11,R13,R15 Solution-stack link. S10 Closure evidence. Versioned/current architecture, supported uses, limitations, failure modes, subgroup evidence, guardrails, update/audit commitments Pass / re-open. Evidence is current enough for deployed version; stale/voluntary gaps remain explicit Re-open when: Model version/update or stale file evidence.

Response to Question 25: Foundation Model Device Master Files — Voluntary Is Insufficient The concept of voluntary Foundation Model Device Master Files (MAFs) is structurally sound but will not function as intended on a voluntary basis. Foundation model developers — large technology companies and AI research organizations — have powerful competitive incentives to withhold training data provenance, model architecture details, and known failure mode documentation. Voluntary disclosure, in the absence of any regulatory consequence for non- disclosure, will produce a MAF system that the most responsible foundation model developers participate in, while the least transparent actors opt out entirely. This creates a perverse dynamic in which transparency is penalized. WHM recommends that FDA establish a conditional market access mechanism: devices built on foundation models for which no MAF has been filed, or for which MAF content falls below FDA's minimum disclosure standards, should face a higher premarket evidence burden — specifically, requiring more extensive sponsor- conducted benchmarking and third-party validation to compensate for the lack of foundation model transparency. This creates a positive economic incentive for foundation model developers to file complete MAFs without mandating disclosure as a legal matter — a framework that is consistent with FDA's existing least- burdensome principles while addressing the practical incentive problem. When filed, MAFs should include, at minimum: a training data provenance summary describing data sources, curation methodology, and known demographic representation gaps; a documented inventory of known failure modes in clinical contexts, including clinical domains, subpopulations, and task types where performance degradation has been observed; performance benchmarks across demographic subgroups for relevant clinical tasks; a model version history with meaningful changelog documentation; guardrail architecture description; and an update notification protocol specifying how and when manufacturers using the foundation model will be informed of material changes.