Ever Fortune.AI Co. Ltd. (Ti-Hao Wang)

See attached file(s)

​PUBLIC COMMENT TO THE U.S. FOOD AND DRUG ADMINISTRATION​
​Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper​
​and Request for Feedback​

​Submitted electronically via Regulations.gov​

​Docket No. FDA-2026-N-7874​

​Date: October 19, 2026​

​Dear CDRH Staff:​

​On​ ​behalf​ ​of​ ​Ever​ ​Fortune.AI,​ ​Co.,​ ​Ltd.,​ ​we​ ​appreciate​ ​the​ ​opportunity​ ​to​ ​provide​ ​comments​ ​on​
​CDRH's​ ​discussion​ ​paper,​ ​Considerations​ ​for​ ​the​ ​Regulation​ ​of​ ​Generative​ ​AI-Enabled​ ​Medical​
​Devices.​​Ever​​Fortune.AI​​is​​a​​Taiwan-based​​medical​​AI​​company​​with​​15​​FDA​​510(k)​​clearances​​for​
​AI-enabled​​medical​​devices​​and​​extensive​​experience​​engaging​​with​​FDA​​throughout​​the​​submission​
​and​ ​review​ ​process.​ ​These​ ​clearances​ ​include​ ​Seg​ ​Pro​ ​V3​ ​(K251306)​ ​with,​ ​to​ ​our​ ​knowledge,​ ​the​
​first​ ​FDA-authorized​ ​PCCP​​for​​radiotherapy​​auto-contouring​​software.​​Drawing​​on​​this​​experience,​
​we​ ​offer​ ​the​ ​following​ ​responses​ ​to​ ​selected​ ​questions​ ​raised​ ​in​​the​​discussion​​paper,​​organized​​by​
​question number.​

​We​ ​recognize​ ​that​ ​GenAI-enabled​ ​medical​ ​devices​ ​introduce​ ​new​ ​challenges​ ​in​ ​performance​
​evaluation,​ ​clinical​ ​reliability,​ ​and​ ​lifecycle​ ​management.​ ​We​ ​strongly​ ​support​ ​rigorous,​ ​clinically​
​meaningful​ ​evidence​ ​requirements​ ​that​ ​ensure​ ​patient​ ​safety​ ​and​ ​device​ ​effectiveness.​ ​At​​the​​same​
​time,​ ​we​ ​believe​ ​that​ ​clear,​ ​scientifically​ ​justified,​ ​and​ ​risk-proportionate​ ​regulatory​ ​pathways​ ​are​
​essential​ ​to​ ​support​ ​responsible​ ​innovation.​ ​Drawing​ ​on​​our​​experience​​in​​developing​​FDA-cleared​
​medical​ ​AI​ ​devices,​ ​we​ ​hope​ ​to​ ​contribute​ ​practical​ ​perspectives​ ​that​ ​help​ ​bridge​ ​regulatory​
​expectations and real-world device development.​

​We​ ​generally​ ​support​ ​the​ ​proposed​ ​two-axis​ ​risk​ ​framework​ ​and​ ​competency-based​ ​evaluation​
​approach.​ ​We​ ​encourage​ ​CDRH​ ​to​ ​ensure​ ​that​ ​evaluation​ ​requirements​ ​are​ ​proportionate​ ​to​ ​the​
​device's​ ​intended​ ​use,​​clinical​​risks,​​and​​potential​​impact​​on​​patient​​safety,​​with​​an​​emphasis​​on​​the​
​safety and effectiveness of the final medical device.​

​We​ ​also​ ​encourage​ ​CDRH​ ​to​ ​provide​ ​clear​ ​and​ ​flexible​ ​regulatory​ ​expectations​​for​​multimodal​​AI​
​systems,​ ​including​ ​vision-language​ ​models​ ​(VLMs),​ ​clinical​ ​confirmation,​ ​and​ ​lifecycle​
​management.​ ​Such​ ​an​ ​approach​ ​would​ ​maintain​ ​rigorous​ ​safety​ ​and​ ​effectiveness​ ​standards​ ​while​
​providing predictable regulatory pathways that support responsible innovation.​
 ：
​●​ ​Q1​ ​Does​​the​​two-axis​​risk​​framework,​​organized​​around​​device​​activity​​and​​the​​consequence​
​of​ ​relying​ ​on​ ​an​ ​incorrect​ ​output,​ ​appropriately​ ​capture​ ​the​ ​dimensions​ ​most​ ​relevant​ ​to​ ​the​
​risk​ ​of​ ​a​ ​GenAI-enabled​ ​software​ ​function?​ ​If​ ​there​ ​are​ ​additional​ ​dimensions—such​ ​as​ ​the​
​reversibility​​of​​a​​resulting​​action,​​the​​availability​​of​​downstream​​safeguards,​​the​​time​​pressure​
​of​ ​the​ ​deployment​ ​setting,​ ​or​ ​the​ ​traceability​ ​of​ ​the​ ​output​ ​(i.e.,​ ​to​ ​primary​ ​source​
​materials)—that​ ​should​ ​be​ ​represented​ ​in​ ​a​ ​risk​ ​framework,​ ​please​ ​describe​ ​and​ ​provide​
​examples of how they should be represented.​

​○​ ​We​ ​support​ ​the​ ​proposed​ ​two-axis​ ​risk​ ​framework​ ​as​ ​a​ ​practical​ ​starting​ ​point​ ​for​
​assessing GenAI-enabled medical devices.​

​We​​recommend​​considering​​the​​intended​​clinical​​workflow​​and​​the​​effectiveness​​of​​human​
​oversight​​as​​additional​​risk-modifying​​factors.​​For​​example,​​GenAI​​functions​​that​​provide​
​clinical​ ​information​ ​for​​independent​​healthcare​​professional​​review​​may​​present​​different​
​risks​ ​from​ ​those​ ​that​ ​directly​ ​initiate​ ​clinical​ ​actions.​ ​These​ ​distinctions​ ​should​ ​be​
​considered when assessing the potential consequences of incorrect outputs.​

：
​●​ ​Q2​ ​CDRH​ ​seeks​​input​​on​​how​​to​​account​​for​​the​​spectrum​​of​​GenAI-enabled​​informational​
​functions​​that​​vary​​in​​the​​degree​​to​​which​​they​​direct​​a​​user​​to​​a​​particular​​action​​(i.e.,​​between​
​“non-directive”​ ​and​ ​“action-directing”).​ ​What​ ​characteristics​ ​of​ ​an​ ​output—such​ ​as​ ​its​
​wording,​​specificity,​​personalization,​​or​​context—could​​be​​considered​​as​​modifiers​​of​​the​​risk​
​associated​ ​with​ ​an​ ​informational​ ​function,​ ​after​ ​accounting​ ​for​ ​the​ ​device’s​ ​overall​
​functionality​ ​and​ ​intended​ ​use?​ ​What​ ​additional​ ​information​ ​would​ ​provide​ ​manufacturers​
​with​ ​sufficient​ ​clarity​ ​and​ ​predictability​ ​around​ ​risk​ ​assessment​ ​for​ ​informational​ ​functions​
​while​ ​recognizing​ ​that​ ​directiveness​ ​may​ ​exist​ ​along​ ​a​ ​continuum​ ​rather​ ​than​ ​as​ ​a​ ​binary​
​distinction?​

​○​ ​We​ ​recommend​ ​that​ ​FDA​​assess​​the​​directiveness​​based​​on​​output​​content,​​intended​​use,​
​and how information is delivered within the clinical workflow.​

​For​ ​example,​ ​“A​ ​pulmonary​ ​nodule​ ​is​ ​observed”​ ​and​ ​“A​ ​follow-up​ ​CT​ ​scan​ ​is​
​recommended”​​differ​​in​​explicit​​directiveness,​​but​​their​​risk​​also​​depends​​on​​presentation,​
​opportunities for verification, and expected clinician reliance.​

​We​ ​encourage​ ​FDA​ ​to​ ​provide​ ​illustrative​ ​examples​ ​explaining​ ​how​​safeguards,​​such​​as​
​source​​traceability​​and​​meaningful​​clinician​​review,​​can​​mitigate​​directiveness-related​​risk​
​and,​ ​when​ ​demonstrated​ ​effectively,​ ​reduce​ ​additional​ ​evidence​ ​requirements.​ ​These​
​examples should provide predictable design and validation expectations for manufacturers.​

​2​
 ：
​●​ ​Q4​ ​CDRH​​seeks​​input​​on​​whether​​and​​how​​the​​distinction​​between​​generalist​​and​​specialist​
​physicians​ ​might​ ​inform​ ​the​ ​assessment​ ​of​ ​risk​ ​for​ ​HCP-facing​ ​GenAI-enabled​ ​functions.​
​Under​​what​​circumstances,​​if​​any,​​might​​risk​​be​​affected​​when​​an​​HCP​​who​​lacks​​the​​relevant​
​clinical​​specialist​​knowledge​​receives​​information​​that​​falls​​within​​a​​specialist​​area​​of​​practice?​
​What device characteristics or safeguards might mitigate such risks?​

​○​ ​We​ ​recommend​ ​that​ ​FDA​ ​consider​ ​differences​ ​in​ ​clinical​ ​expertise​ ​only​ ​where​ ​they​ ​are​
​relevant​​to​​the​​device's​​intended​​use​​and​​the​​user's​​ability​​to​​appropriately​​evaluate​​and​​act​
​on GenAI-generated information.​

​Risks​​may​​arise​​when​​specialized​​outputs​​are​​used​​for​​clinical​​decisions​​beyond​​the​​user's​
​expertise​ ​without​ ​appropriate​ ​safeguards.​ ​These​ ​risks​ ​can​ ​be​ ​addressed​ ​through​ ​clearly​
​defined​​intended​​users,​​appropriate​​labeling,​​training,​​and​​clinical​​review​​mechanisms,​​as​
​applicable.​

​We​ ​encourage​ ​FDA​ ​to​ ​leverage​ ​existing​ ​risk​ ​management​ ​and​ ​usability​ ​engineering​
​frameworks​ ​rather​ ​than​ ​presume​ ​that​ ​separate​ ​validation​ ​across​ ​physician​ ​specialties​ ​is​
​necessary for all GenAI-enabled devices.​

：
​●​ ​Q7​ ​Is​ ​the​ ​competency-based​ ​approach​ ​described​​above,​​i.e.,​​device​​benchmarking​​followed​
​by​ ​clinical​ ​confirmation,​ ​a​ ​useful​ ​and​ ​appropriate​ ​framework​ ​for​ ​evaluating​ ​GenAI-enabled​
​devices?​

​○​ ​We​ ​support​ ​the​ ​proposed​ ​framework​ ​of​ ​device​ ​benchmarking​ ​followed​ ​by​ ​clinical​
​confirmation​​as​​a​​useful​​and​​appropriate​​approach​​for​​GenAI-enabled​​devices,​​provided​​it​
​is​​applied​​to​​the​​final​​device​​as​​configured​​for​​deployment​​and​​tailored​​to​​intended​​use​​and​
​risk.​

​The​ ​two​ ​components​ ​answer​ ​different​ ​questions.​ ​Benchmarking​ ​characterizes​ ​defined​
​competencies​ ​at​ ​scale​ ​under​ ​controlled​ ​conditions,​ ​while​ ​clinical​ ​confirmation​​addresses​
​whether​ ​that​ ​performance​ ​holds​ ​under​ ​real​ ​or​ ​clinically​ ​representative​ ​inputs,​ ​users,​​and​
​workflows.​ ​We​ ​recommend​ ​that​ ​the​ ​two​ ​be​ ​linked:​ ​failure​ ​modes​ ​and​ ​residual​
​uncertainties​ ​identified​ ​in​ ​benchmarking​ ​should​ ​determine​ ​what​ ​clinical​ ​confirmation​
​needs​ ​to​ ​examine,​ ​so​ ​that​ ​confirmation​ ​is​ ​targeted​ ​rather​ ​than​ ​generic.​ ​Where​ ​evidence​
​addresses​ ​both​ ​objectives,​ ​FDA​ ​should​ ​allow​ ​it​ ​to​ ​support​ ​both​ ​components​ ​without​
​duplicative testing.​

​Satisfactory​ ​benchmark​ ​performance​ ​should​ ​be​ ​supported​ ​by​ ​appropriate​ ​clinical​
​confirmation,​ ​with​ ​the​ ​study​ ​design​ ​and​ ​level​ ​of​ ​evidence​ ​determined​ ​by​ ​the​ ​device's​
​intended​​use,​​clinical​​claims,​​and​​associated​​risks.​​FDA​​should​​clarify​​when​​retrospective​
​evaluation,​ ​clinician​ ​adjudication,​ ​or​​other​​approaches​​are​​sufficient.​​We​​encourage​​FDA​
​to let sponsors justify the scope and sufficiency of each component for their intended use.​

​3​
 ：
​●​ ​Q8​ ​How​​might​​the​​two-axis​​risk​​framework​​described​​in​​Section​​IV​​be​​considered​​within​​the​
​competency-based​ ​approach​ ​described​ ​above​ ​to​ ​help​ ​determine​ ​the​ ​level​​of​​evidence​​needed​
​for a premarket submission?​

​○​ ​We​ ​support​ ​using​ ​the​​two-axis​​risk​​framework​​as​​a​​heuristic​​to​​inform​​risk-proportionate​
​premarket evidence requirements.​

​The​ ​framework​ ​should​ ​help​ ​guide​ ​the​ ​selection​ ​and​ ​extent​​of​​benchmarking​​and​​clinical​
​confirmation,​ ​considering​ ​the​ ​device's​ ​intended​ ​use,​ ​clinical​ ​risk,​ ​and​ ​residual​
​uncertainties.​ ​We​ ​encourage​ ​FDA​ ​to​ ​provide​ ​illustrative​ ​examples​ ​to​ ​support​ ​consistent​
​and predictable application of this approach.​

：
​●​ ​Q9​ ​Would​ ​a​ ​benchmarking​ ​structure​ ​such​ ​as​ ​the​ ​one​ ​described​ ​above​ ​be​ ​likely​ ​to​ ​provide​
​adequate​ ​evidence​ ​of​ ​clinical​ ​knowledge,​ ​analytic​ ​capabilities,​ ​safety​ ​behavior,​
​communication,​ ​and​ ​generalizability​ ​to​ ​support​ ​a​ ​reasonable​ ​assurance​ ​of​ ​safety​ ​and​
​effectiveness?​​Are​​there​​elements​​that​​are​​missing,​​redundant,​​or​​inappropriately​​categorized?​
​Are there externally developed standards that could be leveraged?​

​○​ ​We​ ​support​ ​the​ ​proposed​ ​benchmarking​ ​structure,​ ​with​ ​the​ ​applicability​ ​of​ ​individual​
​elements​ ​determined​ ​by​ ​the​​final​​device's​​intended​​use,​​clinical​​functions,​​and​​associated​
​risks, supported by appropriate scientific justification.​

​For​​multimodal​​devices,​​including​​vision-language​​models​​(VLMs),​​we​​encourage​​FDA​​to​
​clarify​​how​​existing​​benchmarking​​elements​​may​​be​​adapted​​to​​evaluate​​modality-specific​
​risks, without necessarily requiring separate benchmarking categories.​

：
​●​ ​Q10​ ​CDRH​ ​recognizes​ ​that​ ​publicly​ ​available​ ​benchmarking​ ​assets​ ​may​ ​be​ ​subject​​to​​data​
​contamination,​ ​saturation,​ ​and​ ​limited​ ​real-world​ ​representativeness.​ ​How​ ​should​ ​a​ ​sponsor​
​establish​ ​that​ ​performance​ ​on​ ​a​ ​given​ ​benchmark​ ​predicts​ ​safe​ ​and​ ​effective​ ​real-world​
​behavior​​for​​the​​device’s​​intended​​use?​​What​​evidence​​should​​support​​the​​construct​​validity​​of​
​a​ ​benchmark​ ​used​ ​to​ ​gate​ ​device​ ​evaluation,​ ​and​ ​what​ ​role​ ​should​ ​sponsor-developed​
​benchmarks play given potential concerns around independence and optimization to the test?​

​○​ ​We​ ​recommend​ ​that​ ​benchmark​ ​validity​ ​be​ ​assessed​ ​based​ ​on​ ​its​ ​relevance​ ​to​ ​the​ ​final​
​device's​ ​intended​ ​clinical​ ​task,​ ​including​ ​the​ ​representativeness​ ​of​ ​test​ ​data,​
​appropriateness of reference standards, and clinical relevance of performance measures.​

​We​ ​support​ ​the​ ​use​ ​of​ ​sponsor-developed​ ​benchmarks​ ​when​ ​publicly​ ​available​
​benchmarks​ ​do​​not​​adequately​​reflect​​the​​device's​​intended​​use.​​Manufacturers​​should​​be​
​able​ ​to​ ​justify​ ​their​ ​validity​ ​through​ ​appropriate​ ​clinical​ ​and​ ​methodological​ ​evidence,​
​with safeguards against data contamination and overfitting to the test set.​

​4​
 ：
​●​ ​Q11​ ​CDRH​ ​is​ ​considering​ ​that​ ​clinical​ ​confirmation​​for​​a​​GenAI-enabled​​device​​might​​not​
​require​ ​a​ ​prospective​ ​clinical​ ​study​ ​in​ ​every​ ​case,​ ​and​ ​has​ ​described​ ​above​ ​a​ ​range​ ​of​
​approaches​​of​​increasing​​rigor​​and​​patient​​exposure.​​How​​might​​a​​sponsor​​select​​and​​justify​​a​
​confirmation​​approach​​tailored​​to​​a​​device’s​​intended​​use​​and​​proportionate​​to​​the​​device’s​​risk​
​profile?​ ​Are​ ​there​ ​device​ ​types​ ​or​ ​risk​ ​profiles​ ​for​ ​which​ ​one​ ​or​ ​more​ ​of​ ​these​ ​approaches​
​would​ ​be​ ​insufficient​ ​or​ ​inappropriate?​ ​How​ ​might​ ​the​ ​anticipated​ ​distribution​ ​of​​real-world​
​inputs​​be​​taken​​into​​consideration?​​Are​​there​​other​​methods​​of​​clinical​​confirmation​​that​​might​
​help inform the evaluation of GenAI-enabled devices?​

​○​ ​We​ ​recommend​ ​selecting​ ​clinical​ ​confirmation​ ​methods​ ​based​ ​on​ ​the​ ​final​ ​device's​
​intended​ ​use,​ ​clinical​ ​claims,​ ​degree​ ​of​​autonomy,​​risk​​profile,​​and​​residual​​uncertainties​
​identified​ ​through​​benchmarking.​​Manufacturers​​should​​justify​​how​​the​​selected​​methods​
​adequately address the relevant clinical questions.​

​For​ ​devices​ ​providing​ ​information​ ​subject​ ​to​ ​independent​ ​clinician​ ​review,​ ​retrospective​
​clinical​ ​evaluation​ ​may​ ​be​ ​appropriate​ ​for​ ​certain​ ​output​ ​accuracy​ ​claims.​ ​However,​ ​for​
​devices​ ​that​ ​directly​ ​influence​ ​time-critical​ ​clinical​ ​decisions​ ​or​ ​autonomously​ ​initiate​
​clinical​​actions,​​evaluation​​of​​device​​outputs​​alone​​may​​be​​insufficient.​​Additional​​clinical​
​evaluation​ ​should​ ​be​ ​determined​ ​by​ ​the​ ​potential​ ​impact​ ​on​ ​patient​ ​safety​ ​and​ ​the​
​uncertainties that remain.​

​Clinical​ ​confirmation​ ​should​ ​consider​ ​the​ ​anticipated​ ​real-world​ ​input​ ​distribution,​
​including​​the​​intended-use​​population,​​clinical​​settings,​​and​​relevant​​input​​characteristics.​
​Representative​ ​premarket​ ​datasets​ ​may​ ​be​ ​supplemented​ ​by​ ​targeted​ ​evaluation​ ​of​
​uncommon​ ​but​ ​clinically​ ​significant​ ​scenarios.​ ​Where​ ​sufficient​ ​premarket​ ​evidence​​has​
​established​​a​​reasonable​​assurance​​of​​safety​​and​​effectiveness,​​postmarket​​real-world​​data​
​may​ ​further​ ​characterize​ ​variations​ ​in​ ​input​ ​distributions​ ​and​ ​support​ ​continued​
​performance evaluation across different clinical environments.​

​We​ ​encourage​​FDA​​to​​allow​​scientifically​​justified​​combinations​​of​​clinical​​confirmation​
​methods,​​including​​staged​​evaluation​​where​​appropriate,​​and​​to​​clarify​​how​​manufacturers​
​may establish the sufficiency of evidence for different clinical functions and risk profiles.​

：
​●​ ​Q14​ ​For​ ​open-ended​ ​device​ ​outputs,​ ​how​ ​should​ ​performance​ ​comparators​ ​and​ ​acceptance​
​criteria​​be​​selected?​​When​​a​​panel​​of​​qualified​​clinicians​​serves​​as​​the​​comparator,​​how​​should​
​the​​applicable​​standard​​(for​​example,​​the​​standard​​of​​care​​versus​​the​​performance​​of​​a​​median​
​clinician​ ​in​ ​practice)​ ​be​ ​defined​ ​and​ ​justified?​ ​Should​ ​generalist​ ​or​ ​specialist​ ​physicians​ ​be​
​used​ ​as​ ​a​ ​performance​ ​standard?​ ​When​ ​should​ ​human-AI​ ​team​ ​performance,​ ​rather​ ​than​​the​
​device operating alone, serve as the basis for evaluation?​

​○​ ​Q14-1 Selecting comparators and acceptance criteria​

​Comparators​ ​should​ ​reflect​ ​the​ ​clinical​ ​task​ ​and​ ​intended​ ​claims.​ ​Established​ ​clinical​
​reference​ ​standards​ ​should​ ​be​ ​used​ ​where​ ​available.​ ​For​ ​open-ended​ ​outputs,​ ​qualified​
​clinical​ ​experts​ ​may​ ​assess​ ​performance​ ​using​ ​predefined​ ​criteria.​ ​Clinically​ ​significant​

​5​
 ​errors​ ​should​ ​be​ ​evaluated​ ​separately​ ​rather​ ​than​ ​obscured​ ​by​ ​aggregate​ ​performance​
​metrics.​

​○​ ​Q14-2 Standard of care versus median clinician​

​Standard​ ​of​ ​care​ ​is​ ​generally​ ​appropriate​ ​for​ ​evaluating​ ​clinical​ ​correctness,​ ​while​
​representative​ ​clinician​ ​performance​ ​may​ ​be​ ​more​​relevant​​when​​assessing​​improvement​
​over​ ​current​ ​practice.​ ​Where​ ​no​ ​definitive​ ​reference​ ​exists,​ ​expert​ ​adjudication​ ​may​
​establish acceptable responses.​

​○​ ​Q14-3 Generalist versus specialist​

​Specialist​ ​assessment​ ​may​ ​establish​ ​clinical​ ​correctness,​ ​while​ ​the​ ​relevant​ ​practice​
​comparator​​should​​reflect​​the​​intended​​user​​population.​​For​​example,​​a​​device​​intended​​to​
​assist generalists may appropriately be compared against unaided generalist performance.​

​○​ ​Q14-4 Human-AI team versus device alone​

​Device-level​ ​evaluation​ ​should​ ​establish​ ​the​ ​reliability​ ​of​ ​generated​ ​outputs.​ ​Human-AI​
​team​ ​evaluation​ ​may​ ​additionally​ ​be​​warranted​​when​​clinical​​benefit​​claims​​involve​​user​
​performance or when significant interaction risks remain.​

​We​ ​encourage​ ​FDA​ ​to​ ​provide​ ​illustrative​ ​examples​ ​clarifying​ ​comparator​​selection​​and​
​when human-AI evaluation is necessary.​

：
​●​ ​Q15​ ​Are​​there​​ways​​in​​which​​performance​​might​​be​​assessed​​relative​​to​​the​​care,​​technology,​
​or​​course​​of​​action​​likely​​to​​occur​​in​​the​​absence​​of​​the​​device,​​rather​​than​​to​​the​​comparators​
​described​​in​​this​​section?​​What​​approaches​​might​​be​​used​​to​​identify​​and​​justify​​a​​comparator​
​such as unaided clinical judgment, delayed specialist review, or no intervention?​

​○​ ​We​ ​support​ ​the​ ​use​ ​of​ ​existing​ ​clinical​ ​practice​ ​in​ ​the​ ​absence​ ​of​ ​the​ ​device​ ​as​ ​an​
​appropriate comparator, where relevant to the device's intended use and clinical claims.​

​Comparators​ ​may​ ​include​ ​unaided​ ​clinician​ ​performance​ ​or​ ​existing​ ​clinical​ ​workflows,​
​supported​ ​by​ ​representative​ ​clinical​ ​data.​ ​The​ ​selection​ ​of​ ​comparators​​and​​performance​
​acceptance​ ​criteria​ ​should​ ​be​ ​aligned​ ​with​ ​the​ ​device's​​intended​​use,​​clinical​​claims,​​and​
​associated​​risks.​​Different​​performance​​objectives,​​including​​superiority​​or​​non-inferiority,​
​may be appropriate depending on the clinical context.​

​Clinically​​significant​​errors​​should​​also​​be​​evaluated​​to​​ensure​​that​​demonstrated​​benefits​
​do not compromise patient safety.​

：
​●​ ​Q16​ ​Should​ ​independent​ ​third​ ​parties​ ​be​ ​involved​ ​in​ ​some​ ​or​ ​all​ ​aspects​ ​of​ ​a​
​competency-based​ ​approach,​​including​​device​​benchmarking​​and​​clinical​​confirmation?​​If​​so,​
​in​​what​​ways​​might​​qualified,​​independent​​third-party​​participation​​contribute​​to​​this​​approach,​

​6​
 ​and​ ​what​ ​qualifications​ ​and​ ​independence​ ​criteria​ ​should​ ​apply?​ ​Are​ ​there​ ​aspects​ ​of​ ​the​
​assessment​ ​for​ ​which​ ​third-party​ ​involvement​ ​would​ ​be​ ​impractical​ ​or​ ​inadvisable?​ ​What​
​safeguards​ ​or​ ​program​ ​design​ ​features​ ​would​ ​be​ ​critical​ ​to​ ​prevent​ ​third-party​ ​participation​
​from limiting competition or preventing innovation?​

​○​ ​We​ ​support​ ​risk-based,​ ​optional​ ​participation​ ​of​ ​qualified​ ​independent​ ​third​ ​parties​ ​in​
​benchmarking​​and​​clinical​​confirmation,​​particularly​​where​​independent​​datasets​​or​​expert​
​adjudication would strengthen the evaluation.​

​Third​ ​parties​ ​should​​demonstrate​​relevant​​technical​​or​​clinical​​expertise,​​appropriate​​data​
​governance,​ ​and​ ​independence​ ​from​ ​the​ ​device​ ​manufacturer.​ ​However,​ ​third-party​
​involvement​​may​​be​​impractical​​for​​assessments​​requiring​​access​​to​​proprietary​​systems​​or​
​close integration with clinical workflows.​

​We​ ​encourage​ ​FDA​ ​to​ ​clarify​ ​when​ ​independent​ ​third-party​ ​involvement​ ​would​
​meaningfully​ ​strengthen​ ​the​ ​credibility​​of​​benchmarking​​and​​clinical​​confirmation,​​while​
​recognizing appropriately justified manufacturer-conducted evaluations.​

：
​●​ ​Q17​ ​CDRH​ ​is​​exploring​​whether​​a​​competency-based​​approach​​could​​be​​applied​​to​​devices​
​that​ ​incorporate​ ​a​ ​variety​ ​of​ ​underlying​ ​model​ ​architectures,​ ​such​ ​as​ ​multimodal​
​vision-language​ ​models​ ​and​ ​generative​ ​or​ ​predictive​ ​world​ ​models.​ ​Are​ ​there​ ​aspects​ ​of​ ​the​
​described​​approach​​that​​might​​be​​ineffective​​or​​inapplicable​​to​​these​​devices?​​What​​additional​
​or distinct considerations might need to be addressed?​

​○​ ​We​​believe​​the​​proposed​​competency-based​​framework​​can​​be​​applied​​to​​different​​GenAI​
​architectures,​ ​including​ ​vision-language​ ​models​ ​(VLMs),​ ​without​ ​requiring​ ​separate​
​benchmarking​ ​frameworks​ ​for​ ​each​ ​model​ ​type.​ ​Evaluation​ ​should​ ​focus​ ​on​ ​the​​clinical​
​competencies,​ ​safety,​ ​and​ ​effectiveness​ ​of​ ​the​ ​final​ ​medical​ ​device,​ ​rather​ ​than​ ​its​
​underlying model architecture.​

​For​ ​VLM-based​ ​medical​ ​devices,​ ​existing​ ​benchmarking​ ​elements​ ​can​ ​be​ ​adapted​ ​to​
​evaluate​ ​multimodal-specific​ ​risks,​ ​including​ ​inconsistencies​ ​between​ ​image​ ​and​ ​text​
​inputs,​ ​unsupported​ ​or​ ​omitted​ ​clinical​ ​findings,​ ​and​ ​performance​ ​variations​ ​associated​
​with​ ​image​ ​quality​ ​or​ ​input​ ​format.​ ​Such​ ​evaluations​ ​should​ ​rigorously​ ​assess​​clinically​
​significant failure modes and their potential impact on patient safety.​

​As​ ​medical​ ​AI​​evolves​​from​​task-specific​​models​​toward​​multimodal​​foundation​​models,​
​we​ ​encourage​ ​FDA​ ​to​ ​establish​ ​architecture-independent,​ ​competency-based​ ​evaluation​
​principles,​ ​supported​ ​by​ ​practical​ ​examples​ ​and​ ​clearly​ ​defined​ ​clinical​ ​performance​
​expectations.​ ​This​ ​approach​ ​would​ ​promote​ ​rigorous​ ​and​ ​consistent​ ​safety​ ​evaluation​
​while​ ​providing​ ​manufacturers​ ​with​ ​a​ ​predictable​ ​regulatory​ ​pathway​ ​for​ ​future​
​innovations.​

​7​
 ：
​●​ ​Q18​ ​CDRH​ ​is​ ​considering​ ​whether​ ​it​ ​may​ ​be​ ​appropriate​ ​to​ ​accept​ ​greater​ ​premarket​
​uncertainty​​regarding​​a​​GenAI-enabled​​device’s​​benefit-risk​​profile​​through​​greater​​reliance​​on​
​postmarket​ ​monitoring.​ ​Under​ ​what​ ​conditions​ ​might​ ​such​ ​an​ ​approach​ ​be​ ​appropriate,​ ​and​
​what​ ​characteristics​ ​of​ ​a​ ​monitoring​ ​program​ ​would​ ​need​ ​to​ ​be​ ​in​ ​place​ ​to​ ​justify​ ​reduced​
​premarket​​evidence?​​Are​​there​​device​​types​​or​​risk​​profiles​​for​​which​​this​​approach​​would​​not​
​be appropriate?​

​○​ ​We​ ​support​ ​a​ ​risk-based​ ​approach​ ​that​ ​allows​ ​defined​ ​residual​ ​uncertainties​ ​to​ ​be​
​addressed​ ​through​ ​postmarket​ ​monitoring,​ ​provided​ ​that​ ​sufficient​ ​premarket​ ​evidence​
​establishes a reasonable assurance of safety and effectiveness.​

​For​ ​example,​ ​this​ ​approach​ ​may​ ​be​ ​appropriate​ ​for​ ​assessing​ ​performance​ ​variations​
​following​ ​deployment​ ​across​ ​healthcare​ ​institutions​ ​with​ ​different​ ​imaging​ ​equipment,​
​patient​ ​populations,​ ​or​ ​clinical​ ​workflows,​ ​where​ ​the​ ​device's​ ​core​ ​clinical​ ​performance​
​has already been adequately validated.​

​Postmarket​ ​evidence​ ​could​ ​help​ ​characterize​ ​these​ ​real-world​ ​variations​ ​and​ ​confirm​
​continued​ ​performance​ ​in​ ​clinical​ ​practice.​ ​However,​ ​this​ ​approach​ ​may​ ​be​ ​less​
​appropriate​ ​where​ ​unresolved​ ​uncertainties​ ​involve​ ​core​ ​clinical​ ​performance​ ​or​ ​where​
​errors could cause serious or irreversible harm before they can be detected and mitigated.​

：
​●​ ​Q19​ ​Please​ ​comment​ ​on​ ​the​ ​potential​ ​approaches​ ​to​ ​postmarket​ ​performance​ ​evaluation,​
​including​ ​periodic​ ​re-benchmarking,​ ​sample-based​ ​clinician​ ​review,​ ​and​ ​performance​
​degradation​ ​monitoring.​​What​​additional​​approaches​​should​​CDRH​​consider,​​and​​how​​should​
​the cadence and triggering events for reassessment be determined?​

​○​ ​We​ ​support​ ​a​ ​risk-proportionate​ ​approach​ ​to​ ​postmarket​ ​performance​ ​monitoring,​ ​using​
​periodic​ ​re-benchmarking,​ ​sample-based​ ​clinical​ ​review,​ ​and​ ​performance​ ​degradation​
​monitoring as appropriate to the device's intended use and risk profile.​

​Reassessment​ ​frequency​ ​should​ ​be​ ​determined​ ​based​ ​on​​the​​severity​​and​​detectability​​of​
​potential​ ​clinical​ ​harm,​ ​observed​ ​performance​ ​stability,​ ​and​ ​the​ ​extent​ ​of​ ​residual​
​uncertainty.​ ​An​ ​initial​ ​reassessment​ ​schedule​ ​may​ ​be​ ​established​ ​based​ ​on​ ​these​ ​factors​
​and adjusted as real-world performance evidence becomes available.​

​Event-triggered​​reassessment​​should​​be​​considered​​when​​monitoring​​identifies​​an​​increase​
​in​ ​clinically​ ​significant​ ​errors,​ ​meaningful​ ​changes​ ​in​ ​input​ ​data​ ​distributions​ ​that​ ​may​
​affect​ ​device​ ​performance,​ ​or​ ​evidence​ ​of​ ​performance​ ​degradation.​ ​The​ ​scope​ ​of​
​reassessment​ ​should​ ​be​ ​proportionate​ ​to​ ​the​ ​potential​ ​clinical​ ​impact​ ​of​ ​the​ ​identified​
​issue, rather than requiring comprehensive revalidation for every monitoring signal.​

：
​●​ ​Q20​ ​Please​ ​comment​ ​on​​whether​​the​​proposed​​approaches​​to​​postmarket​​monitoring​​can​​be​
​facilitated​​by​​machine-based​​supervisory​​agents.​​What​​considerations,​​including​​the​​evaluation​

​8​
 ​and​ ​reliability​ ​of​ ​the​ ​supervisory​ ​agent​ ​itself,​ ​should​ ​CDRH​ ​take​ ​into​ ​account​ ​for​ ​such​ ​an​
​approach?​

​○​ ​We​ ​support​ ​the​ ​use​ ​of​ ​machine-based​ ​supervisory​ ​agents​ ​as​ ​an​ ​optional​ ​approach​ ​to​
​postmarket monitoring of GenAI-enabled medical devices.​

​CDRH​ ​should​ ​consider​ ​evaluating​ ​the​ ​reliability​ ​of​ ​supervisory​ ​agents​ ​based​ ​on​ ​their​
​intended​ ​monitoring​ ​functions​ ​and​ ​the​ ​clinical​ ​consequences​ ​of​ ​missed​ ​or​ ​incorrectly​
​flagged​ ​errors.​ ​Relevant​​evidence​​may​​include​​their​​ability​​to​​detect​​clinically​​significant​
​failures,​ ​false-negative​ ​and​ ​false-positive​ ​rates,​ ​and​ ​performance​ ​across​ ​representative​
​real-world​ ​conditions.​ ​Where​ ​appropriate,​ ​evaluation​ ​should​ ​include​ ​independent​
​reference​ ​assessments​ ​and​ ​review​ ​of​ ​both​ ​flagged​ ​and​ ​unflagged​ ​outputs​ ​to​ ​identify​
​potentially missed errors.​

​The​ ​level​ ​of​ ​validation​ ​and​ ​ongoing​ ​monitoring​ ​should​ ​be​ ​proportionate​ ​to​​the​​extent​​to​
​which​​manufacturers​​rely​​on​​supervisory​​agents​​for​​safety​​monitoring.​​We​​encourage​​FDA​
​to​ ​clarify​ ​how​ ​the​ ​intended​ ​monitoring​ ​function,​ ​associated​ ​clinical​ ​risks,​ ​and​ ​available​
​safeguards​ ​should​ ​inform​ ​the​ ​validation​ ​and​ ​reliability​ ​requirements​ ​for​ ​supervisory​
​agents.​

：
​●​ ​Q21​ ​What​ ​roles​ ​might​ ​clinicians,​ ​healthcare​ ​institutions,​ ​professional​ ​societies,​
​standards-setting​ ​bodies,​ ​and​ ​other​ ​stakeholders​​appropriately​​play​​in​​postmarket​​monitoring,​
​and how can these roles be structured without diffusing manufacturer accountability?​

​○​ ​For​​deployed​​medical​​AI,​​manufacturers​​often​​cannot​​access​​patient-level​​clinical​​data​​or​
​directly observe how clinicians use AI outputs.​

​FDA​ ​should​ ​consider​ ​a​ ​shared-responsibility​ ​framework​ ​distinguishing​ ​manufacturer​
​monitoring​ ​obligations​ ​from​ ​healthcare​ ​institutions'​ ​responsibilities​ ​for​ ​clinical​​oversight​
​and data governance.​

：
​●​ ​Q22​ ​CDRH​ ​envisions​ ​that​​a​​manufacturer’s​​premarket​​competency-based​​assessment​​might​
​serve​ ​as​ ​a​ ​baseline​ ​against​​which​​post-deployment​​modifications​​could​​be​​re-evaluated.​​How​
​might​ ​the​ ​extent​ ​of​ ​re-benchmarking​ ​or​ ​other​ ​evidence​ ​be​ ​scaled​ ​to​ ​the​ ​nature​​and​​expected​
​impact​ ​of​ ​a​ ​given​ ​modification?​ ​For​ ​example,​ ​are​ ​there​ ​categories​ ​of​ ​change​ ​that​ ​might​ ​not​
​significantly​ ​affect​ ​safety​ ​or​ ​effectiveness​ ​of​ ​a​ ​GenAI-enabled​ ​device,​ ​or​ ​might​ ​be​
​appropriately​ ​managed​ ​within​ ​a​ ​sponsor’s​ ​quality​ ​management​​system​​versus​​requiring​​FDA​
​premarket​ ​review​ ​and​ ​authorization​ ​before​ ​implementation?​ ​Are​ ​there​ ​categories​ ​of​ ​changes​
​that might be appropriate for inclusion in a PCCP?​

​○​ ​We​ ​support​ ​a​ ​risk-​ ​and​ ​impact-based​ ​approach​ ​to​ ​re-evaluating​ ​modifications​ ​to​
​GenAI-enabled​ ​medical​ ​devices.​ ​The​ ​scope​ ​of​ ​re-benchmarking​ ​and​ ​other​ ​supporting​
​evidence​ ​should​ ​be​ ​determined​ ​through​ ​a​ ​documented​ ​change-impact​ ​assessment,​

​9​
 ​considering​ ​the​ ​functions,​ ​competencies,​ ​and​ ​clinical​ ​risks​ ​potentially​ ​affected​ ​by​ ​the​
​modification.​

​Changes​ ​that​ ​do​ ​not​ ​materially​ ​affect​ ​clinical​ ​performance,​ ​intended​ ​use,​ ​or​​the​​device's​
​safety​ ​and​ ​effectiveness​ ​may​ ​be​ ​appropriately​ ​managed​ ​under​​the​​manufacturer's​​quality​
​management​ ​system,​ ​with​ ​documented​ ​impact​ ​assessments​ ​and​ ​targeted​ ​verification​ ​or​
​regression testing.​

​Modifications​ ​that​ ​are​ ​sufficiently​ ​defined​ ​in​ ​advance,​ ​including​ ​their​ ​boundaries,​
​validation​​methods,​​and​​acceptance​​criteria,​​may​​be​​suitable​​for​​inclusion​​in​​an​​authorized​
​PCCP.​ ​Changes​ ​that​ ​introduce​ ​new​ ​clinical​ ​functions,​ ​materially​ ​alter​ ​clinical​
​performance,​​or​​fall​​outside​​an​​authorized​​PCCP​​should​​be​​assessed​​under​​applicable​​FDA​
​submission requirements.​

：
​●​ ​Q23​ ​How​ ​might​ ​PCCP​ ​concepts​ ​or​ ​other​ ​change-control​ ​approaches​ ​be​ ​adapted​ ​for​
​GenAI-enabled​ ​devices​ ​when​ ​the​ ​nature​ ​or​ ​scope​ ​of​ ​future​ ​modifications​ ​cannot​ ​be​ ​fully​
​prespecified?​

​○​ ​We​ ​encourage​ ​FDA​ ​to​ ​consider​ ​competency-based​ ​PCCPs​ ​for​ ​GenAI-enabled​ ​medical​
​devices,​ ​allowing​ ​controlled​ ​evolution​ ​of​ ​underlying​ ​model​ ​architectures​ ​while​
​maintaining​​predefined​​clinical​​functions,​​safety​​boundaries,​​and​​performance​​acceptance​
​criteria.​

​As​ ​medical​ ​AI​ ​evolves​ ​from​ ​task-specific​ ​models​ ​toward​ ​foundation​ ​models,​ ​future​
​modifications​​may​​involve​​changes​​in​​model​​architecture,​​training​​methods,​​or​​underlying​
​foundation models that cannot be fully anticipated at the time of initial authorization.​

​Rather​​than​​requiring​​every​​technical​​implementation​​detail​​to​​be​​predetermined,​​a​​PCCP​
​could​​define​​the​​permitted​​scope​​of​​modifications,​​the​​clinical​​competencies​​that​​must​​be​
​maintained,​ ​and​ ​the​ ​validation​ ​methods​ ​and​​acceptance​​criteria​​necessary​​to​​demonstrate​
​continued safety and effectiveness.​

​We​ ​recognize​ ​that​ ​such​ ​flexibility​ ​must​ ​be​ ​supported​ ​by​ ​rigorous​ ​change-impact​
​assessments,​ ​clinically​ ​meaningful​ ​validation,​ ​and​ ​appropriate​ ​safeguards​ ​against​
​cumulative​ ​performance​ ​degradation.​ ​Changes​ ​outside​ ​the​ ​authorized​ ​PCCP​ ​boundaries​
​should remain subject to applicable FDA submission requirements.​

​We​ ​encourage​ ​FDA​ ​to​ ​provide​ ​practical​ ​examples​ ​illustrating​ ​how​ ​competency-based​
​PCCPs​​could​​support​​responsible​​model​​evolution​​while​​maintaining​​regulatory​​oversight​
​and patient safety.​

​10​
 ：
​●​ ​Q24​ ​For​ ​devices​ ​built​ ​on​ ​third-party​ ​foundation​ ​models,​ ​changes​ ​to​ ​the​ ​underlying​ ​model​
​may​​be​​initiated​​by​​the​​third-party​​model​​developer​​rather​​than​​the​​device​​manufacturer.​​How​
​can​ ​a​ ​manufacturer​ ​detect,​ ​evaluate,​ ​and​ ​respond​ ​to​ ​such​ ​changes​​in​​a​​timely​​manner?​​What​
​mechanisms—for​ ​example,​ ​contractual,​ ​technical,​ ​or​ ​through​ ​a​ ​PCCP—could​ ​provide​
​reasonable​​assurance​​that​​third-party​​developer-initiated​​changes​​do​​not​​compromise​​the​​safety​
​or effectiveness of the device?​

​○​ ​We​ ​support​ ​a​ ​risk-​ ​and​ ​impact-based​ ​approach​ ​to​ ​managing​ ​changes​ ​to​ ​third-party​
​foundation​ ​models​ ​(FMs).​ ​The​ ​scope​ ​of​ ​re-evaluation​ ​following​ ​an​ ​update​ ​to​ ​an​
​underlying​​foundation​​model​​should​​be​​determined​​through​​a​​documented​​change-impact​
​assessment,​​considering​​potential​​effects​​on​​the​​final​​device's​​clinical​​performance,​​safety,​
​and effectiveness.​

​Manufacturers​ ​may​ ​use​​a​​combination​​of​​provider​​notifications,​​version​​information,​​and​
​performance​ ​monitoring​ ​to​ ​identify​​potential​​upstream​​changes.​​However,​​their​​ability​​to​
​detect​ ​and​ ​manage​ ​such​ ​changes​ ​may​​depend​​on​​the​​level​​of​​control​​over​​the​​third-party​
​FM​ ​and​ ​access​ ​to​ ​change-related​ ​information,​ ​including​ ​whether​ ​model​ ​versions​ ​can​ ​be​
​fixed, updates can be controlled, and change information is available.​

​When​ ​a​ ​change​ ​is​ ​identified​ ​or​ ​suspected,​ ​its​ ​potential​ ​impact​ ​should​ ​be​ ​assessed​​at​​the​
​final-device​ ​level​​to​​determine​​whether​​targeted​​testing​​is​​sufficient​​or​​broader​​validation​
​is warranted.​

​We​​encourage​​FDA​​to​​consider​​these​​practical​​limitations​​when​​clarifying​​expectations​​for​
​third-party​ ​FM​ ​change​ ​management,​ ​while​ ​maintaining​ ​flexibility​ ​in​ ​the​ ​selection​ ​of​
​appropriate verification and validation approaches.​

：
​●​ ​Q25​ ​Would​​voluntary​​Foundation​​Model​​MAFs​​be​​practical​​for​​foundation​​model​​developers​
​to​​provide​​and​​to​​keep​​current,​​and​​what​​content​​might​​they​​include?​​Given​​that​​participation​
​would​ ​be​ ​voluntary​ ​and​ ​that​ ​model​ ​developers​ ​may​ ​have​ ​limited​ ​incentive​ ​to​ ​disclose​
​safety-relevant​​information,​​what​​would​​make​​such​​a​​program​​sufficiently​​useful​​for​​premarket​
​review?​ ​Are​ ​there​ ​alternative​ ​mechanisms​ ​CDRH​ ​should​ ​consider​ ​for​ ​obtaining​ ​information​
​about underlying foundation models?​

​○​ ​We​ ​support​ ​the​ ​establishment​ ​of​ ​voluntary​ ​Foundation​ ​Model​ ​Master​ ​Files​ ​(MAFs),​
​allowing​ ​medical​ ​device​ ​manufacturers,​​particularly​​small​​and​​medium-sized​​enterprises,​
​to​​reference​​relevant​​technical​​and​​safety​​information​​submitted​​by​​third-party​​foundation​
​model developers.​

​This​​approach​​could​​reduce​​duplicative​​documentation​​efforts​​while​​protecting​​proprietary​
​model​ ​information.​ ​We​ ​encourage​ ​FDA​ ​to​ ​clarify​ ​the​ ​procedures​ ​for​ ​referencing​ ​and​
​maintaining​ ​MAFs​ ​and​ ​consider​​mechanisms​​that​​incentivize​​participation​​by​​foundation​
​model​ ​developers,​ ​such​ ​as​ ​reducing​ ​redundant​ ​information​ ​requests​ ​during​ ​premarket​
​review.​

​11​
​Respectfully submitted,​

​Ti-Hao Wang​
​Chief Technology Officer​
​Ever Fortune.AI, Co., Ltd.​
​8F., No.360, Sec. 1, Jingmao Rd., Beitun Dist.​
​Taichung City, 406040​
​Taiwan​
​tihao.wang@everfortune.ai​

​12​