On October 24th, 2023, U.S. Food and Drug Administration (FDA), Health Canada, and U.K Medicines and Healthcare products Regulatory Agency (MHRA) published 5 guidance principles for predetermined change control plans (PCCPs). These principles are designed to guide the development of PCCPs to ensure any changes to artificial intelligence/machine learning-enabled medical devices (MLMD) do not pose inappropriate or further risk to the user. The document also provides insight into how developers may build and maintain engagement with users and other stakeholders on PCCP concepts for MLMD. Specifically, the regulatory bodies call for more transparency between developers and the users of their MLMD with respect to changes being made to the MLMD. The joint release of this guidance aims to harmonize international standards relating to plan changes in the machine learning space.
Previously, in 2021, FDA, Health Canada, and MHRA composed a guidance of 10 principles that educate developers on Good Machine Learning Practice (GMLP). These principles touch on how developers can ensure their devices which incorporate machine learning are safe, high quality, and effective for users. The original 10 principles can be found here:
Good Machine Learning Practice for Medical Device Development: Guiding Principles | FDA
These regulatory bodies utilized the previously published GLMP principles to model the new five principles related to MLMD, specifically principle 10. Principle 10 states “Deployed models are monitored for performance and-retraining risks are managed” (FDA, Health Canada, MHRA, 2021). This principle directly informed the development of new PCCP guidelines because it touches on monitoring performance and mitigating re-training risks in existing devices. PCCPs are generally outlined by the manufacturer. PCCPs may aid in mapping regulatory processes with rapid, dynamic approaches to managing changes in MLMD. Additionally, control plans mitigate and control risks in an effective manner via inspection, maintenance, and possibly through better device performance.
Changes to MLMD can be basic to complex, such as changes to an algorithm or to models. Since changes to MLMD range across a broad spectrum from normal to significant, changes may require regulatory oversight through subsequent premarket reviews. However, the time taken for such a plan may not coincide with the fast-paced growth of MLMD. This is where PCCPs may be helpful as the regulatory bodies recommend that PCCPs should outline plans for device modification, carrying out and controlling such modifications, and analyzing modification impacts. The regulatory bodies outline crucial considerations in the 5 guiding principles that emphasize characteristics of successful PCCPs to establish the groundwork for future plans.
Five Guiding Principles
- Focused and Bounded
The first principle of the guidance provides strategies for focusing and bounding the PCCP. Changes described in PCCP should be limited to modifications that are defined in the intended use of the device or the intended purpose of the MLMD. This may include the degree of planned changes, changes within the scope of the MLMD, assessment of the planned changes, and plans for changing the device within the parameters of the PCCP. Plans for changing the device can include how to conduct the changes or how to stop implementing the changes if there is excessive risk or the need for the change is no longer applicable.
- Risk-based
The second principle of the guidance aims to ensure that the development of the PCCP is risk-based. Assessing risk at every stage of development enhances the value and reliability of the PCCP. This should be implemented throughout the total product lifecycle (TPLC) as it provides assurance that changes, either a single change or multiple, remain applicable and appropriate for the MLMD and its users.
- Evidence-based
The third principle focuses on evidence-based changes, involving the implementation of changes based on feedback received. This evidence should be generated throughout the TPLC. Evidence may help developers demonstrate the safety and effectiveness of devices with PCCPs, that benefits outweigh the risks, and that any existing risks are properly managed. When determining methods for collecting evidence, developers should include scientific justification of metrics used to measure performance of the MLMD as well as plans for weighing the benefits against risks both before and after the change is implemented in accordance with the PCCP.
- Transparent
The fourth principle of the guidance concentrates on developers being transparent with information to users and stakeholders. Transparency requires that stakeholders know of the performance of the MLMD before and after changes. Similarly, stakeholders should be made aware of the type of data being used to create the product with justification as to why it’s relevant for the population that is intended to use it, types of tests that will be run for changes that are planned, and if the characterization of the device will change after changes are made. Users and stakeholders should also stay informed on how the device will be monitored, how problems will be detected and how any problems or deviations will be resolved. This could all be accomplished using meticulous plans outlining all the information mentioned above in the PCCP.
- Total Product Lifecycle (TPLC) Perspective
The last principle focuses on PCCPs being based on the lifecycle of the device. Considering the TPLC could reinforce the quality of a product’s PCCP since it should lead to implementing risk management and incorporating the viewpoint of users and stakeholders. This dynamic principle also encourages developers to consistently incorporate existing regulatory, quality, and risk management strategies. This, in turn, establishes a system for monitoring the device, reporting problems, and answering any issues with safety on a continual basis.
Conclusion
The intention of these principles is to help manage certain device changes where regulatory approval before marketing may be required. All 5 guidance principles are available to view on the Software as A Medical Device (SaMD) section FDA.gov. FDA is also accepting comments and feedback on these principles via the FDA public docket (FDA-2019-N-1185)
Unsure how to develop a PCCP that may be subjected to regulatory oversight in the future? MEDIcept is here to help you understand and implement all 5 of the guiding principles in your MLMD PCCP. Contact us today at sales@medicept.com.
References (cited)
Health Canada, U.K. Medicines and Healthcare products Regulatory Agency, U.S. Food and Drug Administration (2023). Good Machine Learning Practice for Medical Device Development: Guiding Principles. Published by Authors.
Health Canada, U.K. Medicines and Healthcare products Regulatory Agency, U.S. Food and Drug Administration (2023) Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles. Published by Authors.
Elizabeth Smith – Associate Medical Device Consultant