Hattangadi Shilpa

Comments are attached in file below.

Thank you for the discussion paper and for the opportunity to comment. I am a physician-scientist with
26+ years across clinical practice, public (NIH) biomedical funding, hematology research, and digital
health advisory work. I am an advisor to and cofounder with equity in GoodHart Labs, a company that
makes an independent run-time verifier for clinical AI. The views in this comment are still explicitly my
own and are not made on behalf of GoodHart Labs or any other organization.

My comments address Section VI and Question 18.
Premarket validation alone is not sufficient. Open-ended models cannot be tested against every input
they will ever meet, and after deployment, they fail in ways that premarket validation does not reveal:
model drift, silent failure when ground truth is unavailable, subgroup and equity gaps, and vendor
changes that the health system cannot see. Post-deployment monitoring is a reasonable addition,
provided it is specific and frequent enough to catch those failures.

The discussion paper lists monitoring approaches, including periodic re-benchmarking, sample-based
clinician review, and degradation monitoring. It does not define who should run them, how independent
they should be, how often they must occur, or what happens when they flag a problem. Health systems
often adopt AI because they are short of staff and/or expertise, which is also why they cannot monitor
clinical AI performance continuously. Without a defined standard, post-deployment monitoring becomes
an added risk rather than an assurance.
Recommendation 1: Require post-deployment monitoring. Where premarket validation is reduced
under Question 18, require vendors to submit a monitoring plan that includes: prespecified
performance thresholds for clinically relevant subgroups, a defined cadence plus triggering events
(including foundation model updates), named accountability, and explicit criteria for stopping use. To be
honest, periodic checks alone are not enough given the failure modes listed above. Even the UK has
recognized the risk of intermittent monitoring and accepted all 44 recommendations of its National
Commission into the Regulation of AI in healthcare including “continuous real-world monitoring”. I
would recommend the same: post-deployment continuous surveillance of clinical AI output.

Recommendation 2: Keep review independent. Question 16 and Section V.D.2 raise the right issues
about independent review panels and segregated data. For post-deployment surveillance,
independence should depend on funding, ownership, and data access, not only on the absence of
formal employment. A vendor should not be the sole evaluator of its own deployed tool.
Recommendation 3: Share responsibility explicitly. In response to Question 21, the vendor should
remain accountable for performance claims. Deploying institutions should document local readiness:
onboarding validation, oversight, monitoring capacity, and who responds to alerts. Professional
societies can contribute reference guidelines for ground truth.
Recommendation 4: Keep verification affordable. Small developers and under-resourced
community hospitals will bear the highest costs of AI oversight. Shared, dynamic benchmark resources
and an accredited third-party pathway would make verification feasible beyond large health systems.
Otherwise, continuous monitoring becomes available only to well-resourced systems, which are often
not the ones that most need these tools.

The useful question is not whether these devices should be verified after deployment, but how that
verification is designed into the tool from the start. This discussion paper is well placed to make that
explicit, and I would welcome the opportunity to contribute to the next draft.