Regulating Generative AI Medical Devices: A Medicept Expert Roundtable

Generative AI is already finding its way into medical devices: tools that summarize patient records, answer patient questions, draft clinical documentation, and increasingly take actions on their own across multiple steps. The technology is moving faster than the regulatory framework built to evaluate it.

In August 2026, FDA’s Center for Devices and Radiological Health (CDRH) released a discussion paper, Considerations for the Regulation of Generative AI-Enabled Medical Devices, and opened a public docket (FDA-2026-N-7874) asking how the agency should approach risk, premarket evidence, postmarket monitoring, third-party foundation models, and agentic AI. Comments are due October 19, 2026.

MEDIcept’s quality, regulatory, and clinical specialists worked through the paper question by question to prepare our comments. Along the way, they didn’t always agree. We asked five of them to weigh in on what FDA got right, where the framework needs work, and what device makers should be doing now.

Meet the panel

• Gregg Van Citters, 
  Principal Consultant, Quality Engineering
• Kanchana Iyer,  Principal Consultant, Regulatory 
• Melissa DeHaas, Director, Regulatory
• Mikir Bodalia,  Digital Quality Principal Consultant 
• Ryan Satterly, Director, Quality Assurance

First, what is the FDA Proposing?

The discussion paper is not guidance and creates no requirements. It lays out FDA’s early thinking and poses a series of questions for public feedback. The core ideas:

• A two-axis risk framework that places a GenAI function’s activity (from providing information to acting autonomously) against the consequence of relying on an incorrect output.
• A competency-based premarket approach, in which sponsors identify the competencies a device needs for its intended use and show them through nonclinical benchmarking and, where needed, clinical confirmation.
• Lifecycle monitoring through re-benchmarking, clinician review, and possibly machine-based supervisory agents.
• Early positions on foundation models and agentic AI, including a voluntary Foundation Model Device Master File (MAF) that model developers could submit directly to FDA.
• Start from intended use, and build benchmarks and evidence that trace back to it.
• Map the full workflow from GenAI output to clinical action, and design oversight that works under real conditions.
• Test for consistency, not just accuracy: repeatability and reproducibility matter.
• Plan evidence in proportion to risk, using retrospective, simulated, and shadow-mode methods where they answer the question.
• Treat foundation models as critical suppliers, with both contractual and technical controls.
• Build change control and postmarket monitoring in from day one, ideally as a continuous process.

Does GenAI need a whole new regulatory approach?

Ryan Satterly: No, and that was the thread running through all of our comments. GenAI brings real new problems: probabilistic behavior, open-ended conversations, dependence on models you don’t own, and systems that can act on their own. But risk management, human factors, design controls, supplier controls, and change management still apply. They need to be adapted, not thrown out. The question FDA should keep asking is the one it has always asked: is the finished device safe and effective in the workflow where it will actually be used?

Gregg Van Citters: I’d agree the principles hold. Where I’d push is on how much faith we put in the technology underneath. Large language models mimic human language without any real understanding of it. We should build the framework with that in mind, not assume the model “knows” what it’s saying.

Is FDA’s two-axis risk framework enough?

Gregg Van Citters: It’s a good start, but what FDA calls “activity” is really a compound variable. Degree of oversight, safeguards outside the GenAI portion of the device, latency: those all act as throttles on activity. People are going to want guidance on how many of those dimensions there are, how to scale them, and how to combine them into something meaningful. I’m not a fan of RPN, but that’s the kind of practical question manufacturers will ask.

It helps to borrow from ISO 14971 here. The GenAI output maps loosely to P1, the probability of a technological failure. P2 is the chance that the hazardous situation actually leads to harm, and that’s where a clinician, a narrowly trained checking agent, or traditional deterministic software can step in. The emerging AI-specific guidance, such as AAMI TIR34971, points the same way.

Mikir Bodalia: Right. Two devices can do the same task, with the same worst-case harm, and carry very different real-world risk. If one recommendation is reviewed by a clinician, backed by sources, and checked by a software safety gate, and another is accepted with one click in a busy workflow, the odds that an error reaches the patient aren’t the same. We asked FDA to keep the two axes but name reversibility, time to intervene, independent verifiability, detectability, and downstream safeguards as explicit risk modifiers.

We also asked FDA to look beyond the GenAI function itself. The same output could be shown to a clinician as advice, dropped straight into an order, handed to another algorithm, or acted on by an agent. Identical output, very different risk. Evaluation has to follow the whole path from output to clinical action, including interoperability with other systems.

Can an output be correct and still cause harm?

Melissa DeHaas: Yes, and this is one of the places we asked FDA to broaden its thinking. The paper frames consequence around reliance on an incorrect output. But a medically accurate statement delivered with more confidence than the evidence supports, a correct answer that leaves out a contraindication, or a constant stream of low-value alerts that trains people to stop looking can all lead to unsafe decisions. We recommended FDA frame it as reliance on device behavior that could lead to an unsafe decision, not just wrong answers.

Gregg Van Citters: GenAI’s propensity for overtrust belongs in the risk estimate. These outputs are fluent and persuasive. That’s a hazard in its own right.

Is “human in the loop” a real safeguard?

Melissa DeHaas: Only if the human can actually catch the problem. A clinician who sees an authoritative-sounding recommendation, without supporting evidence, under time pressure, as one of hundreds that day, isn’t providing much protection. We asked FDA to separate nominal human involvement from effective oversight. Does the user have the information and expertise to judge the output? Is there time to intervene? Does the design encourage real review, or passive acceptance?

Gregg Van Citters: And that expertise has to come from education, training, experience, and consultation with other experts. It can’t come from a foundation model. If the reviewer’s knowledge is coming from the same kind of system they’re reviewing, the check isn’t independent.

Melissa DeHaas: Conversational devices add another layer. A chatbot might say a symptom has several causes, then say the user’s description fits one, then describe how it’s usually managed, and finally suggest monitoring at home seems reasonable. Each reply sounds careful. Together, the patient hears a recommendation. We asked FDA to judge how directive a device is across the whole conversation, and to test realistic paths where users push for specifics or try to pull the device outside its intended scope.

What makes a GenAI benchmark meaningful?

Mikir Bodalia: It has to trace back to intended use. A device that summarizes oncology records might score well on a general medical Q&A benchmark, and that tells you almost nothing about whether it catches the oncology details that matter or keeps a patient’s timeline straight. We recommended the same logic as design controls: intended use, then use scenarios, then hazards, then required competencies, then evaluation methods, then acceptance criteria, then clinical confirmation where needed.

On who builds the benchmark, we told FDA not to assume public benchmarks are more credible. A public benchmark can be independent and still clinically irrelevant. A well-built sponsor benchmark, with sequestered test data and blinded scoring, can be stronger evidence. What matters is whether it actually measures the competency it claims to.

Gregg Van Citters: I’d add repeatability and reproducibility, for both the device and the benchmark. If the model gives different answers each time, or different answers for different users, it may not be useful at all, let alone safe. Given similar inputs, the benchmark should produce consistent ratings too.

How much clinical evidence will GenAI devices need?

Kanchana Iyer: This is the point we feel most strongly about. FDA’s paper says clinical confirmation “might not require a prospective clinical study in every case,” and we agree. Using GenAI shouldn’t automatically trigger a prospective study. The evidence should scale with the uncertainty left after nonclinical testing and with how serious a failure would be.

A tool that summarizes information for clinician review may be well supported by rigorous retrospective evaluation and human factors validation. A system that autonomously selects or starts a high-risk treatment is a different story. We proposed a step-up path: retrospective or simulated evaluation, then shadow-mode or real-world observational data, then a prospective study only when the remaining questions require it.

Choosing the right comparator matters just as much. If a tool helps radiologists review prior records faster, the question isn’t whether the AI can beat a radiologist on its own. It’s whether radiologists using the tool do at least as well as radiologists using today’s workflow. We asked FDA to base comparators on what would realistically happen without the device, instead of defaulting to clinician-versus-AI.

What changes after the device is on the market?

Ryan Satterly: GenAI devices can change between review cycles in ways traditional devices don’t: a foundation model update, a new retrieval source, edited prompts or guardrails, a new user population. We recommended combining periodic reassessment with predefined triggers that prompt an earlier look, tied into complaint handling, CAPA, and risk management. And not every change needs the same response. A formatting tweak and a foundation model swap aren’t equivalent, so we proposed that testing scale with the expected impact of the change, with PCCPs defining categories of allowed changes rather than every implementation detail.

Gregg Van Citters: This is where I’d go further than our formal comments. New models come out every few weeks now. A new or modified harness can markedly change how the underlying model behaves, so what look like minor tweaks can have major consequences. It’s a bit like keeping up with the flood of cybersecurity vulnerabilities. I think reassessment should be built into the development lifecycle, something like a CI/CD pipeline, with continuous surveillance feeding back into development. Any change to the foundation model could invalidate your assumptions.

Ryan Satterly: That’s a fair challenge, and I don’t think the two ideas conflict. Impact assessment tells you how much testing a change needs. A continuous pipeline makes that testing fast and routine enough that you actually do it every time.

Can AI be trusted to monitor AI?

Ryan Satterly: It can help, with a big caveat. If the supervisory agent shares a foundation model, training data, or knowledge source with the device it’s watching, it may share the same blind spots and confirm the very errors it’s supposed to catch. Redundancy only lowers risk when the controls are truly independent. Automated monitoring should add to postmarket surveillance, not replace it.

Gregg Van Citters: If you’re going to do it, a group of agents built on different models and harnesses is more likely to work. Ideally it mirrors a normal team, with product development, quality, regulatory, and medical affairs agents questioning each other’s assumptions and working toward consensus. Even then, a human reviewing that activity is still a good idea for the foreseeable future.

How should device makers handle third-party foundation models?

Ryan Satterly: The Foundation Model MAF is a promising idea because it gives FDA information device makers can’t get on their own. But it leaves a gap. The manufacturer is still responsible for the finished device, often with little visibility into, or leverage over, a large model provider’s updates. We asked FDA to define a minimum set of safety-relevant information providers should share with device makers, such as version identification, notice of material updates, known healthcare limitations, and guardrail changes. Where contracts fall short, FDA should recognize technical controls: version pinning, change detection, regression gates before deploying a new version, fallback configurations, and rollback.

What’s different about agentic AI?

Mikir Bodalia: With agents, every step can look right while the overall sequence goes wrong. An agent might correctly identify a clinical issue, choose the right next step, and open the right tool, then enter the wrong parameter, or keep going after a change in the patient’s condition that should have stopped it. Early mistakes compound. We asked FDA to require end-to-end evaluation of the full action sequence: planning, ordering, permissions, checks on intermediate results, knowing when to escalate to a human, and the ability to stop safely. We also suggested tying acceptance criteria to the most clinically consequential action the agent can take, weighed against the chance to intervene before it happens.

Final thoughts: where does GenAI in medical devices go from here?

Gregg Van Citters: My fundamental concern with relying on LLMs to make, or even guide, clinical decisions is that they mimic language without understanding. Add the reproducibility problems in the basic science that underpins clinical knowledge, and there’s real risk of patient harm. I think the best hope lies in world models and classifier models working alongside LLMs. Once those mature, risk management gets simpler and more predictable, because explainability improves and can be tied to actual probabilities from real clinical situations.

Kanchana Iyer: That’s exactly why proportionate evidence matters. It lets low-risk, well-controlled tools reach clinicians sooner while holding higher-risk uses to a higher bar.

Melissa DeHaas: Whatever the model, the device has to be designed around how people actually use it, under real pressure.

Mikir Bodalia: And tested as a system, not as a model on a leaderboard.

Ryan Satterly: None of this requires reinventing medical device regulation. It requires applying what already works, with the rigor GenAI demands.

Key takeaways for companies building GenAI-enabled devices

Put integrated expertise behind your GenAI device

GenAI devices sit where regulatory strategy, quality systems, human factors, and clinical evidence meet. Getting one to market, and keeping it there, takes all of those working together. MEDIcept brings regulatory, quality, and clinical expertise into one team, with hands-on experience across SaMD, AI/ML, and digital health, and more than 600 regulatory submissions supported over 30+ years. Patients are waiting. Talk to our team about building a regulatory and evidence strategy for your GenAI-enabled device.

Frequently asked questions

What is FDA’s discussion paper on generative AI medical devices? It’s a CDRH paper released in August 2026, Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and Request for Feedback. It outlines FDA’s early thinking on risk assessment, premarket evaluation, postmarket monitoring, foundation models, and agentic AI, and asks for public feedback on a series of questions. It is not guidance and does not create requirements.

When are comments due on FDA’s GenAI discussion paper? Comments on docket FDA-2026-N-7874 are due October 19, 2026.

What is FDA’s two-axis risk framework for GenAI devices? It places a GenAI function’s level of activity (from providing information to acting autonomously) on one axis and the severity of harm from relying on an incorrect output on the other. MEDIcept recommends adding risk modifiers such as reversibility, time to intervene, and downstream safeguards.

Will GenAI-enabled medical devices require prospective clinical studies? Not necessarily. FDA’s paper states that clinical confirmation might not require a prospective study in every case. MEDIcept recommends that the level of clinical evidence scale with residual uncertainty and potential harm.

Why do repeatability and reproducibility matter for GenAI devices? GenAI models can return different answers to the same input or for different users. A device that isn’t consistent is hard to validate and may not be safe, so both the device and the benchmarks used to evaluate it should be tested for consistency.

What is a Foundation Model Device Master File (MAF)? It’s a voluntary mechanism FDA is exploring that would let a foundation-model developer share proprietary information directly with FDA. The device manufacturer would remain responsible for the safety and effectiveness of the finished device.

What extra considerations apply to agentic AI medical devices? Because agentic systems carry out multi-step actions, safety can’t be inferred from validating individual steps. Manufacturers should evaluate the complete action sequence, including error propagation, permission boundaries, escalation to humans, and the ability to halt or reverse actions safely.

Sources

• FDA — Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and Request for Feedback
• FDA — FDA Seeks Public Feedback to Inform Regulatory Approach for Generative AI-Enabled Medical Devices
• Arnold & Porter — FDA Seeks Public Feedback on Regulatory Approach for Generative AI-Enabled Medical Devices
• SBA Office of Advocacy — FDA Requests Feedback on Considerations for Generative AI-Enabled Medical Devices Regulation