Article

Reduce Surprises During FDA-Required HF Validation Tests

Ke Li Yew

Published on: June 18th, 2020

The ultimate value of connected medical devices is determined by user engagement. In our experience, ensuring you’ve built something that meets user expectations is best achieved by implementing an iterative release process, built on feedback over time.

The Drawbacks of a Linear Development Process

Often organizations invest a large amount of time into a software release, leaving testing and validation as a wrap-up activity. This linear approach to software development is especially true in the highly regulated digital medical device space, making each product release a coveted event. The extent of industry regulations has, at times, led organizations to fear testing anything but a polished, “perfect” product. Sometimes this reticence to test is driven by fear of documenting high rates of use-related error; other times it is driven by fear of burdensome rigor that will delay timelines. In the end, the practice of infrequent releases can lead to more expensive design updates and increased risks related to user experience. To address this, we typically recommend implementing fast feedback loops.

Build, Measure, Learn

The cadence of such a process can be summarized as a continuous cycle of build-measure-learn. In this process, each component is executed with frequent iterations that are constantly supporting the release process, where incremental changes can be released often, and we can analyze the effect of those changes in real time. Organizations are then able to engage in a rapid, iterative process rather than a linear methodology that holds off on user feedback until the very end.

Take, for example, an insulin pump with physical buttons as part of the interface. Though the designers intended the buttons to be modern and sleek, the design and development team may not have accounted for the high incidence of reduced tactile sensitivity due to peripheral neuropathy in the patient population. It might not be revealed until summative human factors testing that participants fail to begin dosing because they cannot determine if a button has been pressed. Had the organization engaged in frequent releases and fast feedback loops, this likely would have been discovered early on, saving time and money.

Can Your Medical Device Support a Frequent Release Process?

Not all releases are the same; risk analysis will determine the extent to which each release may or may not require additional summative testing. We can leverage software segregation to determine feature or change requirements and assess the use-related impact of those changes through the lens of relevant risk frameworks (e.g. IMDRF SaMD risk framework). Some releases will require additional regulatory filings and potentially more formative and summative human factors testing, while others will not have those requirements. It is worth noting that the FDA’s Software Precertification Pilot Program for Software as a Medical Device (SaMD) builds on the International Medical Device Regulators Forum (IMDRF) framework to define the level of review required for minor or major changes:

Figure 1. Table 4 from the FDA’s Working Model V1.0 of the Software Precertification Pilot Program

While this pilot program is not (yet) a regulatory pathway that can be used for new submissions, it is a useful framework that can be leveraged internally to determine what types of changes can be assessed with fast feedback loops alone. With a frequent release methodology, by the time your organization needs a release that requires summative testing (i.e., human factors validation testing as described in IEC/ISO 62366-1 and FDA guidance on Applying Human Factors and Usability Engineering to Medical Devices among others), you will have already set the stage for success by creating incremental design improvements from prior releases.

With frequent testing throughout the product’s lifecycle, you will have developed an effective testing methodology that is representative of real-world use-scenarios. Ideally, you will have seen users perform these scenarios, have reduced the risk as much as possible, and will not be hit with surprises at the time of validation, which can be costly and negate the summative test.

In short, by implementing an iterative, agile frequent release cycle, you can mitigate risks related to cost and user experience while paving the way for seamless summative human factors testing.

Has your organization embraced frequent releases? How has it impacted your product development process? We’d love to hear from you at [email protected].

How Can I Begin to Implement This Process?

The next blog in our series will give you some basic tools you’ll need to implement frequent releases in your organization.

About the Authors

Bob Moll is the principal UX Architect at Orthogonal. You can email him at [email protected]
Beth Lester is a UX Researcher at Bold Insight. You can email her at [email protected]

About This Blog Series

Given the multidisciplinary nature of MedTech and connected health, we have the opportunity to work with a wide variety of professionals, including engineers of many stripes, scientists, researchers, designers, experts in regulatory, and quality management. Our collaborations with the talented UX and human factors researchers at Bold Insight have been especially synergistic because Bold Insight’s specialization in user research for regulated medical devices is an excellent complement to Orthogonal’s specialization in the design and development of connected mobile medical devices (CMMD) and Software as a Medical Device (SaMD). But more than working on a common niche, Bold Insight and Orthogonal are fellow travelers on the road to slay the same dragon: we both believe that there is a huge opportunity to move the needle on healthcare costs and outcomes through the effective use of digital health solutions.

In recent years, both of our organizations (and clients!) have recognized how research and development teams can work together in a much more seamless and effective manner when they are aligned with a common focus on user-centered design. This is even more true when the R and the D in R&D jointly leverage methods based on fast feedback loops that generate rapid, iterative learning and progress such as Lean UX and Agile software development.

Given our mutual passion for getting safe, engaging, clinically effective, and comparatively effective medical devices to market faster, we thought we’d use this new series of blogs to share from our experience what that process has looked like in this new series of blogs.

Let us know what you think!

**

About Orthogonal

Orthogonal is a software developer for connected mobile medical devices (CMMD) and software as a medical device (SaMD). We work with change agents who are responsible for digital transformation at medical device and diagnostics manufacturers. These leaders and pioneers need to accelerate their pipeline of product innovation to modernize patient care and gain competitive advantage.

Orthogonal applies deep experience in CMMD/SaMD and the power of fast feedback loops to rapidly develop, successfully launch, and continuously improve connected, compliant products—and we collaborate with our clients to build their own rapid CMMD/SAMD development engines. Over the last eight years, we’ve helped a wide variety of firms develop and bring their regulated/connected devices to market.

About Bold Insight

Bold Insight is a user experience and human factors research agency. Designing, managing, and executing projects that span the product development life cycle, we conduct user research informing early product design to global human factors validation. Working with digital, next-generation technology—medical devices to mobile apps, in-car systems to websites, back office systems to end-to-end customer journeys – we specialize in large-scale and global research. https://boldinsight.com