Blending Agile Practices With Industry Regulations: Webinar

It may be hard to believe, but it’s been 10 years since AAMI published TIR:45 2012, Guidance On The Use Of AGILE Practices In The Development Of Medical Device Software. Orthogonal is hosting a series of events and conversations to mark the occasion, aimed at sharing cutting-edge insights that will collectively help us move faster and break nothing in SaMD development.

On July 28th, 2022, we hosted a Q&A webinar with Michael Iglesias and Larkin Lowrey. These two highly successful practitioners of Agile SaMD, who have held roles at Roche, J&J, Eli Lilly, Novo Nordisk, Philips and Tandem Diabetes Care, were the stars of our June 24th webinar. There, they delivered fantastic insights, but didn’t have the time to delve into the great questions asked by our audience. We invited them to return the next month so they could give detailed answers to the attendees’ open questions on blending Agile practices with QMS, IEC 62304 and ISO 13485 at a tactical level. They were joined by Bernhard Kappe, Orthogonal’s CEO and Founder, who gave his perspective, developed across numerous projects, on leading an Agile SaMD and connected medical device systems development firm whose clientele includes PhysIQ, metaMe Health, Medtronic, Boehringer Ingelheim and Google. This webinar was moderated by Randy Horton of Orthogonal. 

As with the June webinar, we were thrilled with this webinar’s level of audience engagement, the caliber of questions asked, the sizable audience who watched the entirety of the “core” webinar and the respectable number of viewers who stayed for the additional Q&A segment. 

What follows in this blog are:

• 5 Key Takeaway Points
• An embedded video of the webinar 
• A summary of the webinar’s conversation

Interested in attending a future webinar in this series? Sign up for email notifications about upcoming events that will help your organization move faster and break nothing with Agile SaMD development.

Five Key Takeaway Points

1. The Agile values of short, discrete cycles are compatible with the work we do in MedTech. Crucial steps in the SaMD development process, such as human factors testing, documentation and verification and validation can all benefit from adopting Agile methodologies. 
2. Risk classification and device class are good benchmarks by which to break down a monolithic medical device software system, but the non-device features of the medical device should receive the same care and attention of those that are. 
3. Documentation needs to be tailored to the audience reading it. Using an Agile-drive, machine-readable language to automate documentation speeds up the process and facilitates the creation of audience-specific versions. It also reduces the need for the parallel manual maintenance of software documentation, software code and regulatory documentation.
4. Human factors activities can be successfully woven into the Agile process. Primarily, frequent, iterative formative testing of in-process code smooths the summative testing that takes place after a code freeze. 
5. Automating design controls and documentation endows developers with confidence and keeps Agile development nimble. 

Webinar Recording

For other events in this series, check out our Move Faster and Break Nothing Webinar Series.

Webinar Summary

This summary of the webinar was edited for brevity and clarity. To hear the full discussion, which includes additional questions and answers, please watch the full recording of the session.

Randy: How do we break down the work we do in MedTech into smaller chunks? How do we ease the tension between big monolithic solutions and nimble Agile processes? Where do we draw the boundaries?

Michael: I worked at one company where we spun up a small team called the software frameworks team. They were responsible for creating common components and features that mobile apps and similar projects could use.

Imagine you wanted to build an insulin pen. You can go and grab all of the plastic parts and springs and similar objects off the shelf and combine them into a device, right? We took a similar approach with software. The software frameworks team built and pre-verified iOS and Android frameworks, so when the product team was ready to build their app, they could grab the components they needed right off the shelf. It’s an example of how a small team put in work that helped us build a bigger product.

Larkin: Getting theoretical for a moment, the purpose of Agile is to produce short and discrete iterations that allow you to learn from small increments, rather than trying to create a moonshot and finding out, two years later, that you missed the mark.  

As part of every iteration, you need to complete the product development cycle. Everything in that short cycle has to be worthy of release to production, even if you don’t end up shipping it. 

When you take the goal of Agile, and then combine it with the scrum’s goal of releasing usable software at each sprint, you have to start thinking in terms of what you can achieve in a single sprint.

Scrum teams are better when they’re smaller. You have to think about the team operating autonomously, and how effective they can be at delivering some unit of functionality. That means pairing things way back, including your validation. You’ll need to make your validation exercises more discreet and focused to make them more achievable. 

Randy: To break down a monolithic system, must you use a risk classification and device class as a way to encapsulate, contain or define modules? 

Bernhard: That’s one of the tools. It’s just good architecture and good practices. I would be judicious about segregation techniques, as it can add additional burden. If your device is a Class 1 device with Class 1 functionality, you don’t need to do the extra segregation. But using risk classification and device class as a guide is a commonly used strategy. It will inform you of what level of granularity you’ll need to do your documentation, risk management activities, risk mitigation and other special controls. 

Larkin: In terms of software architecture practices, employing domain-driven development as a practice helps you better identify what those boundaries are and how to best partition your code from a functional standpoint.

Bernhard: There are no hard and fast rules about it, but in our experience, separating the parts of your medical device from the non-device features of your commercial product is immensely helpful. Your app, web or cloud software might have user alerts or gamification or reordering features, for example, and when these get submitted in a design history file as part of a 510(k), you’ll start to get all kinds of questions. If you draw a line around what is a medical device and what isn’t, your regulators will thank you, and you’ll be glad you did when you update your software further down the line.

Larkin: The distinction I would make from what Bernhard is saying is, if you imagine you’re keeping two sets of books – one with the part of your system that’s regulated, and one with the part that isn’t – you might be only submitting one set of books to the regulators, but you give the same consideration to both.

Randy:  This next question is actually a rollup of a few related questions we’ve received from the audience. I’ll throw them all out at once and let you all decide how you want to parse them out. What happens when well-documented medical device development meets Agile? Is this an inherently conflict-ridden relationship or a symbiotic one? How do you avoid duplicative work when creating a design history file? How do you align regulatory documentation and processes with Agile, which believes that the best documentation is working code? 

Bernhard: Within medical device development, you’re going to need to do documentation, verification and validation and risk management that are not part of the standard Agile process. One of the things you need to be careful with is not having your development methodology baked into your quality management system, because you may want to change it along the way. You want to make sure what’s in your SOPs is generally applicable even if you change your methodology, otherwise you can’t adjust and improve along the way without taking the much bigger step of modifying your QMS. 

Then you have all of the methodology more at a product level. You’re creating the same documentation, you’re just doing it iteratively. If you have the right tooling along the way, you can do it not as a separate activity but as an output of your regular development methodology in an Agile process. 

Michael: Which gets us to the “why.” Why are we doing so much redundant documentation? Can we right-size this in a way that’s not so burdensome? 

Larkin: Right. Documentation needs to be of use. It’s not just homework to do for the regulators. It provides some kind of value. I look at documentation as a means of conveying information so that it can be easily consumed by any reviewer – be it a regulator, an auditor, a product owner or software engineer. 

But documentation written for a software engineer probably isn’t consumable by other human beings. Software engineers consume information differently than those other types of device professionals. The documentation needs to be tailored so that it can be maximally useful (and minimally distracting or confusing) to a regulator or an auditor, for example. 

I’ve talked in a previous webinar about using Gherkin syntax to encode requirements into a machine-readable file format that you can then produce documentation from. Doing so, you’re able to produce documentation in a form that a regulator or reviewer can consume, and for a software engineer or a quality systems engineer to consume.

Randy’s note: We highly recommend that you take a look at the above-linked talk about Gherkin syntax, regardless of what role you play in the medical device lifecycle. It’s a very impressive and illustrative example of how Agile software engineering methods can create huge value and major leaps forward in the process of developing and then enhancing a medical device. 

Randy: Would prompting engineers to add comments to the code would be useful?

Bernhard: Yes, building checklists into your processes and tooling is super helpful. You can also handle it when you do pull requests, if you have a design review by other developers or systems engineers as needed. 

Larkin: I’ve heard complaints from engineers that comments within code become obsolete very quickly because engineers do maintenance on code, not on comments. It’s very hard for a machine to verify that you’ve updated your comments correctly. 

There is another, related approach, which is the ability for most languages to create structured annotations. Where the developer is producing their code, they can annotate the different functions in classes. Then we can generate structure documentation from those annotations, and it’s a bit easier for a machine to verify. 

Randy: Here’s another bundle of related questions for you to tackle together:  How do you integrate human factors into Agile device work? How do you balance the project-specific requirements of human factors within an Agile process? How would you define an Agile human factors summative test? If you want to do more frequent releases, how do risk management standards in ISO 14971 and human factors standards in IEC 62366 integrate into Agile development to allow for frequent and quick launches?

Bernhard: So let’s be clear, and this is a crucial point… you don’t start with an Agile cadence on Day 1. There’s upfront human factors work that may take longer – like task analysis, understanding use errors and how those might fit into preliminary hazard analysis. But we have found that the same Agile practices of small chunks and fast, frequent feedback help reduce risk and increase confidence in UX design and human factors. 

Our development teams have integrated user experience (UX) design and human factors, where designers and developers work together on a sprint cadence. We also leverage the concept of Three User Thursdays, where we test the formative backlog of human factors and design considerations every two weeks with users. You want human factors to stay ahead of development, so you can reduce risk and defect rate when something comes out of design. 

Michael: It’s ideal to balance the project-specific requirements of human factors with an Agile process. I’ve had teams leverage stakeholder demos at the end of each sprint or ideation as an informal kind of human factors or validation activity. We’ve also considered releasing engineering builds to the internal project team, to test as if they were a user and report bugs or errors into the backlog. 

When you get to a build freeze, you can take those same builds and go into a formative human factors study. You can also polish the pre-product, so to speak, and place it in front of real users. But we know that a single human factor study only lasts about a week, and you can’t really spin up a bunch of Agile releases and fixes within that one week. Doing human factors testing in a longer formative cycle, internally and externally, helps that final summative study go smoothly.

Larkin: The Agile process encourages you to release valuable software every sprint. That doesn’t mean that at the beginning of the spirit, the developers go in without knowing anything. You give the developers a story that is baked. You’ve prepared that story for execution, and that preparation involves doing formative studies and potentially prototyping, as Michael described. 

Though Agile pushes you to deliver releasable code at every iteration, you’re not going to get from the back of the napkin sketch to the final release in that short of period of time. It’s a pipeline, where your stories are going through an iterative set of stages and becoming more and more refined until they’re finally ready for developers to implement. It reduces development time from end to end and also enables parallelization. When you’ve completed one formative study, you can move on to the next without waiting on developers. You can run your teams in parallel by keeping this pipeline filled.

Bernhard: As you do formative human factors testing during development, you’re building up a scaffolding that will support your product when it reaches a design freeze and starts formal verification. If you’ve done it right, you should end up with far fewer last-minute surprises. 

Summative human factors testing is the same. Frequent formative testing during development smooths the summative testing process. In traditional development, I’ve seen that teams often don’t do enough human factors testing during the project, and they frequently end up with unpleasant surprises at the end. These consume cycles of time, budget and opportunity cost in terms of a delayed product launch. The whole idea of Agile processes is to root out those unpleasant surprises early and often in the life cycle, where you can adjust to them in a timely and cost-effective manner. Not to mention this is a morale-effective manner for both your team doing the work and your executive sponsors paying for the work.

The need to do summative human factors should be risk-based. Is there patient risk that is sufficient as a result of use errors? In the case of low-risk software, formative testing can be enough. 

Randy: Here’s one last bundle of related questions: How do you use change control or design change procedures to maintain software using Agile methodologies? How would you define regulated, validated Quality Management Systems (QMS) with Developer Operations (DevOps)? How do you handle design input reviews in an Agile setting? What happens with change control when you mix in a product with both medical device and non-medical device functions? 

Bernhard: As you build a more complex system, you need more tooling and automation in your process, otherwise development will slow down. These are things such as configuration management, CI/CD pipelines and test-driven development, and building testing up around that. You need to extend the same thing to design controls and documentation. 

We have found tremendous value in putting in place an electronic QMS (eQMS) integrated with Application Lifecycle Management (ALM)/Product Lifecycle Management (PLM) that works at the level of granularity of the software development process. This allows you to automate so much of the documentation, integrate with the Continuous Integration/Continuous Delivery (CI/CD) pipeline and make sure your process is efficient and free of defects. 

Larkin: If you have comprehensive automation on your testing, you can typically run your full battery of regression tests and requirements verification tests in a short amount of time. If, for example, you’re exploring a change to look for potential consequences, your automation can tell you that answer in a blink. Automation gives you two things: Greater confidence that you know the impact of a change, and the ability to improve the cycle time at the press of a button. 

How does this apply to DevOps and infrastructure? If you take an Infrastructure-as-Code approach, using Terraform by Hashi Corp for example, it allows you to build out and manage your infrastructure using code. It’s code that’s maintaining your Git repository, subject to the same change control processes that you have for source code, as well as reviews, etc. 

In order for your testing to be truly repeatable, you need to be able to build out a brand-new system in a known state. The Infrastructure-as-Code approach guarantees (or at least comes close to guaranteeing) that the environment that you’re running your tests in are at a known state. At the same time, we can produce a validated infrastructure for our system to operate in, and the tools to verify through automation that the infrastructure is in the correct configuration. 

Bernhard: That’s where the automation of documentation with ALM/PLM tools helps. You can quickly see where the impacts of change are without having to do all this manual work. When you extend it to DevOps, think of it as your entire value change: Not just your development or infrastructure, but all your other processes that are needed to make the medical device. Risk management, human factors and requirements documentation processes are all part of that value chain as well, and where ALM/PLM can really help.

Randy: Thank you to our panelists, and to our audience who submitted these meaty questions. We will be doing more webinars and roundtable discussions, so send us your questions. We want to make this a valuable series as we continue it over the rest of the year. 

Want to hear more from Michael Iglesias and Larkin Lowrey? They shared their insights on how to move faster and break nothing with Agile SaMD development in our June 2022 webinar. Click the button below to read a summary of the webinar and watch the video recording.