Roadmap for Generative AI Inside Medical Devices: Webinar

Generative AI has been a massive buzzword across every industry since the release of ChatGPT a mere 17 months ago. Organizations are rushing to develop a wide range of applications for the technology, hoping to transform their businesses and how they serve their customers ahead of their competitors. In MedTech, there’s been significant efforts to apply generative AI to different parts of the industry. However, the discussion on the role of generative AI inside medical devices has been more measured. There are questions as to how this nascent technology would fit under the design controls of ISO 13485 and IEC 62304.

The emerging consensus among Orthogonal and many of our colleagues in the Software as a Medical Device (SaMD) space is that, barring very specific use cases, generative AI is unlikely to be a core component of medical devices anytime soon. This relatively new technology is still too immature and unpredictable to meet the device industry’s benchmarks for safety and effectiveness. However, like other technologies over the last 100+ years, we expect that generative AI will eventually work its way into the functioning of medical devices.

What will it take for generative AI to become a tool in the toolbox of medical device manufacturers, and what does the history of MedTech teach us about how this might happen?

Orthogonal held a webinar on May 21st, 2024 that discussed possible roadmaps for including generative AI inside medical devices. Bernhard Kappe, Orthogonal’s CEO and Founder, and Clay Anselmo, Principal Quality and Regulatory Consultant at Shriner & Associates, discussed how our industry has incorporated other new technologies into medical devices, as well as the specific aspects of generative AI that pose new and interesting challenges to this adoption.

Webinar Recording

Key Takeaway Points

1. For any new technology used in a medical device, manufacturers need to be able to convincingly mitigate its uncertainties so that the benefits of using the technology outweigh the risks. At present, the opacity of Generative AI algorithms, and how they are designed for a seemingly limitless number of use cases, makes it next to impossible to use our traditional validation methods to prove its safety and effectiveness. Rather than being purpose-built to solve specific use cases, Generative AI has unlimited options for inputs and outputs, meaning we can’t ensure it will give a correct answer every time.
2. In contrast, Machine Learning (ML) algorithms currently applied in medical devices are closed systems where the algorithm is “locked” in terms of improvements. They are trained on fixed data sets and verified and validated based on a separate, statistically large set of data not used for training. The FDA has drafted guidance about machine learning algorithms and how often the continuous learning inherent to ML can be fed back into the medical device. No “unlocked” algorithm, where real-time learning and improvements are continuously put back into the medical device in production, has been cleared by the FDA.
3. It’s currently unclear if it would be practical, in terms of ROI, for a device manufacturer to incorporate Generative AI inside medical devices, given the astronomical costs required to conduct the necessary research to prove out new methods of validation. As a result, our speakers don’t see a short-term path to Generative AI being included in 510(k) or even De Novo submissions.
4. The only possible exception to this would be if a medical device incorporating Generative AI was the sole solution to a critical medical issue emerging during a major crisis. For example, the development and approval of mRNA-based vaccines, a new and not broadly proven medical technology, was accelerated by the FDA because it appeared to be the most viable option for combating the global COVID-19 pandemic.
5. Our speakers did see great potential for use cases of Generative AI in the production of a medical device. For example:

a. They imagine AI assistants pointing out where developers missed things in code, or even having AI write the code – as long as it is backed up by robust quality assurance and quality control methods to catch potential problems.
b. They could see Generative AI writing medical device documentation, as it’s already highly template-based, and there exists a large body of work that could be used for training. Our speakers could imagine an AI quickly generating a ~10,000-page long regulatory submission. The resulting documentation would still require extensive human review, but it could save time and potentially lead to a higher quality submission.
c. There’s also the potential to use AI to extrapolate additional test cases or inputs for medical device testing based on a fixed data set, However, due to ChatGPT’s tendency to hallucinate, these inputs and outputs would need to be highly vetted.

6. Finally, our speakers pointed out that Generative AI is subject to the same issues of bias and representativeness of training data as ML. It can only extrapolate based on the data it is given. For example, if your test case data is only of cisgender men, it will not be able to create additional data inputs that accurately reflect cisgender women.

Speakers

Clay Anselmo, Principal Quality and Regulatory Consultant, Shriner & Associates

Clay is an internationally recognized regulatory compliance expert with an unmatched track record in resolving pre- and post-market quality and regulatory compliance issues. Clay has provided strategic input and direction and led complex quality-related projects for hundreds of medical device companies facing regulatory enforcement. Clay regularly provides strategic consulting on complex quality and regulatory matters including quality system development, remediation, regulatory submissions and clinical studies. In addition to his duties at Shriner & Associates, Clay currently provides strategic direction as a board member to a number of life sciences companies across the globe.

Bernhard Kappe, CEO and Founder, Orthogonal

Bernhard Kappe is the Founder and CEO of Orthogonal. For over a decade, Bernhard has provided thought leadership and innovation in the fields of Software as a Medical Device (SaMD), Digital Therapeutics (DTx) and connected medical device systems. As a leader in the MedTech industry, Bernhard has a passion for launching successful medical device software that makes a difference for providers and patients, as well as helping companies deliver more from their innovation pipelines. He’s the author of the eBook Agile in an FDA Regulated Environment and a co-author of the AAMI Consensus Report on cloud computing for medical devices. Bernhard was the founder of the Chicago Product Management Association (ChiPMA) and the Chicago Lean Startup Challenge. He earned a Bachelor’s and Masters in Mathematics from the University of Pennsylvania, and a Bachelor’s of Science and Economics from the Wharton School of Business.

Moderator

Randy Horton, Chief Solutions Officer, Orthogonal

Randy Horton is Chief Solutions Officer at Orthogonal, a software consulting firm that improves patient outcomes faster by helping MedTech firms accelerate their development pipelines for Software as a Medical Device (SaMD), digital therapeutics (DTx) and connected medical device systems. Orthogonal makes that acceleration happen by fusing modern software engineering and product management tools and techniques (e.g., Agile, Lean Startup, User-Centered Design and Systems Thinking) with the regulated focus on device safety and effectiveness that is at the heart of MedTech.

Horton serves as Co-Chair for AAMI’s Cloud Computing Working Group, as well as AAMI CR:510(2021) and the in-process Technical Information Report #115, all of which address how to safely move medical device computing functions into the cloud. He is a frequent speaker at conferences and webinars, including events hosted by AdvaMed, AAMI, HLTH, RAPS and the Human Factors and Ergonomics Society (HFES).