The MedTech AI Factory: From Algorithm Discovery to Compliant Deployment

Randy Horton

Published on: September 15th, 2026

Executive Summary

For MedTech organizations planning to use AI across multiple products or features, developing a successful model is only the beginning. A capability can perform well and still fail to gain adoption because it disrupts clinical work. Development can stall when the company lacks usable data, and a deployed product can become hard to assess when monitoring arrangements were never established.

Orthogonal’s webinar, The MedTech AI Factory: From Discovery to Deployment, examined how organizations can prepare for these problems before they delay development or limit a product’s usefulness. The AI Factory as a concept connects decisions about what to build, what data is needed, how performance will be demonstrated, and how the product will be deployed and monitored. The goal is to make that process repeatable across multiple algorithms.

The panel brought together perspectives from medical-device software development, AI product development, and software and AI device certification. The discussion focused primarily on machine learning, then considered large language models (LLMs) and Agentic AI. Across both, teams need to decide which healthcare problem to solve, whether they can obtain and use the necessary data, how they will demonstrate that the product works, and how they will evaluate its performance and manage changes after release.

Key Takeaways

1. Build a Repeatable Path From Discovery to Deployment

An AI Factory connects discovery, formal development, regulatory work, deployment, and monitoring so that successive algorithms can build on established processes and tools.

2. Start With Healthcare Value and Workflow Fit

Use early prototypes to understand the patient or provider problem, the existing workflow, and the incentives affecting adoption before committing to a technical solution.

3. Make Data Strategy Part of the Product Roadmap

Plan what information to collect, how to govern it, and what rights are needed to use it for development, validation, future changes, and monitoring.

4. Turn Experiments Into Reproducible Development

Record the data and methods used during discovery, reproduce the work through formal development procedures, and select performance measures that can also be evaluated after deployment.

5. Plan for Monitoring and Change Before Release

Establish how real-world data will become available and how it will support performance assessment and future algorithm modifications.

6. Prepare for Differences Across Regulatory Markets

Examine market-specific AI and data requirements early, recognizing that common development principles do not guarantee identical compliance expectations.

7. Bound LLM Functionality and Broaden Risk Assessment

Define which tasks the LLM can perform, track the model versions being used, and address cybersecurity, data leakage, and intellectual-property risks alongside patient safety.

Why MedTech Needs an AI Factory

An AI Factory gives an organization a repeatable process for developing and releasing multiple algorithms. Those algorithms may operate within a medical device or in the connected software and services. This approach is useful when an organization expects to develop more than one AI capability and needs established ways to collect data, develop and assess algorithms, deploy them, and monitor performance.

Success has two parts. The algorithms need to help patients or providers, fit into their work, and be used in practice. The organization also needs to reduce the time and effort required to develop and release each new algorithm while meeting regulatory and quality expectations.

The process spans data collection and discovery through formal training, assessment, submission, deployment, and monitoring. Decisions made during development affect what must be possible after deployment. For example, a team cannot monitor a performance measure unless the deployed product can provide the information needed to calculate it. Established procedures, reusable tools, and automatically generated records reduce the effort of managing those connections for each new algorithm.

Start With Healthcare Value and Adoption

“Start from the value you want to create in the healthcare system, and the value you want to create, either for patients or for providers, and then work backward from there”

Karl Surmacz, Director of AI and Data Science, Zimmer Biomet

Starting with a prediction target and an accuracy goal can produce a different solution than starting with the healthcare problem the product should address.

Workflow is a practical test of that value. An algorithm can perform well but struggle to gain adoption if it adds steps to a care team’s work or makes an existing task more difficult. Early prototypes give users something concrete to evaluate, helping teams understand both the problem and the constraints of the setting in which the product will operate.

The discovery process needs to answer several questions including:

What patient or provider problem would the capability solve?
Where would it fit into the existing workflow?
Which tasks cannot tolerate additional work or complexity?
Who would use it, benefit from it, and pay for it?
Could helping one stakeholder create cost or work for another?

Healthcare economics affects those answers. The discussion used a case study of externally applied and implanted nerve stimulators to illustrate how a cheaper option with comparable performance may still face adoption barriers. Procedural revenue, existing reimbursement codes, and incentives to keep operating rooms busy can favor the established approach. The example showed that technical performance and price alone don’t explain adoption.

Payment arrangements differ across healthcare systems. The panel contrasted the separate incentives of U.S. insurers, hospitals, and other stakeholders with a system such as the NHS, where an organization may consider both the cost of providing care and the longer-term benefits. However, selling into a more unified healthcare system may give suppliers fewer alternative buyers to approach, which can limit their negotiating power. Teams therefore need to understand who pays, who benefits, and which incentives will affect adoption in the intended market. Discovery may also identify new services that make work easier and could generate revenue or qualify for reimbursement.

Define the Product Before Committing to the Evidence Strategy

Before investing substantial effort in regulatory documentation, teams need reasonable clarity about what the product will do and what benefit it is intended to provide. They can then decide what data, testing, and clinical or performance assessments will show it works as intended. If the product’s intended function changes substantially, the work needed to support it may change too.

Regulatory determination develops alongside discovery and may differ across jurisdictions. One question is whether the proposed capability qualifies as a medical-device function. That determination affects the subsequent development and evidence work. The webinar emphasized that the reasoning behind such decisions should also be supported by records that can withstand review.

The product’s intended use and the evidence needed to support it should guide data science and engineering from the start. A team needs to know which patients and clinical conditions its data should represent, how it will measure performance, and what assessment will support the intended use.

Make Data Strategy Part of the Product Roadmap

The data that a product collects today affects which algorithms the organization can develop in the future. A device may be deployed for years before a team identifies an AI opportunity, only to discover the manufacturer missed the chance to collect valuable variables. (The same problem can arise when a manufacturer wants to monitor a performance measure after launch but lacks the information needed to calculate it.)

A solid data collection strategy needs to address:

Purpose and justification: What information could be collected, and why should it be collected?
Origin and quality: Where did the data come from, how was it collected, and what is known about its quality?
Representation: Does the data include the patients and clinical conditions the product is intended to address?
Permissions: Are institutional review board permissions, investigational device exemptions, or other study requirements relevant?
Reference data: What information will the team compare the algorithm’s results against when assessing performance?
Future use: What data will be needed to validate the algorithm, evaluate later changes, and monitor performance after release?

Useful information may come from outside the device or clinical record. The discussion cited location and weather data as potentially relevant to conditions such as asthma, COPD, and dry eye disease. It also pointed to pharmaceutical companies’ consideration of biomarkers in clinical trials and to CRM data as examples of a broader view of the information an organization holds or could collect.

“You should treat the data as an asset.”

Bernhard Kappe, Founder and CEO, Orthogonal

Making data strategy part of the product roadmap requires weighing collection and maintenance costs, privacy concerns, and whether the information will support a defined development or monitoring need. For a startup, this may begin with small decisions that preserve future options rather than a large infrastructure investment. Waiting can make those same decisions more expensive to implement later.

Establish Data Access and Rights Before They Become Barriers

Identifying useful data does not mean the organization can obtain permission to use it. Manufacturers may struggle to secure enough high-quality data for training and assessment. That becomes especially difficult when a product is intended to work across a wide range of patients, conditions, or uses.

Existing product relationships do not guarantee access either. The discussion included examples of companies having to pay customers for data generated by their own devices because earlier contracts had not anticipated its use for product improvement. Established manufacturers may have valuable relationships and deployed products, but legacy agreements can limit what those advantages let them do.

In 2026, institutions working with medical device manufacturers may expect a benefit in return for providing data. A manufacturer might offer useful information or services to the institution while gaining access to data needed for product development or improvement. Those arrangements need to account for what each organization receives.

Teams need to plan for the cost of obtaining data and generating evidence beyond the initial demonstration of clinical efficacy. Startups may have enough capital to demonstrate clinical efficacy but then need further evidence for reimbursement. Larger organizations face many of the same evidence needs, with different funding horizons. Data-access planning should therefore consider what is needed for development, market entry, reimbursement, and ongoing performance assessment, rather than stopping at the first milestone.

Turn Discovery Into Controlled, Reproducible Development

Exploration tests whether an idea appears promising. Formal development needs to show that the team can reproduce the relevant work using identified data sets, documented methods, and appropriate development procedures. The webinar connected that transition to formal training and Good Machine Learning Practices (GMLP).

Records maintained during exploration make that transition easier. Although exploratory work may not require the same documentation as formal development, teams benefit from being able to explain which data was used and how it changed. Metadata, audit trails, controls over who can access or change data, and records of data handling support that explanation.

Teams need procedures that define how the work is performed and records that show those procedures were followed. Inspections and audits may examine those records, including the reasoning behind whether a capability qualifies as a medical-device function. Automatically generating records during development can reduce the effort of reconstructing them later.

Teams also need to agree on success measures. The choice of machine-learning metric depends on the use case and the consequences of errors, including the relative tolerance for false positives and false negatives. Those choices influence system design, validation, and what the organization will need to monitor in use.

The discussion offered bias monitoring as a concrete example. A team may want to evaluate performance across demographic groups, but the relevant demographic information can be difficult to collect. Teams need to determine whether they need that information, can obtain it appropriately, and can use it to benefit the patient. If the team cannot obtain the demographic information needed for its analysis, it may be unable to evaluate the performance differences it planned to monitor.

The discussion also identified assessment of the algorithm’s clinical performance as a separate activity to consider where appropriate.

Plan for Algorithm Changes Before Release

Teams should consider how they will evaluate future algorithm changes before release. The webinar discussed predetermined change control plans (PCCPs) as a way to plan for anticipated modifications in the U.S. regulatory context. It also raised whether to address a PCCP with the initial submission or separately when the future changes are clearer, and identified this as a topic for pre-submission discussions.

The underlying development questions are practical:

What changes might the organization make to the algorithm?
How will it evaluate those changes and generate supporting evidence?
What data will show whether performance changes as the algorithm encounters different patients or conditions in use?
How will performance be assessed after an update?

Considering those questions before release helps connect the change strategy to data collection, development, and monitoring. A team that expects to improve an algorithm later needs a way to understand how the current version behaves and what information will support the next one.

Build Post-Market Monitoring Into Deployment

Once a product is in use, the manufacturer needs information about whether performance remains consistent with its intended purpose. The challenge includes both obtaining and analyzing that information.

Teams need agreements that permit access to relevant real-world data and a way to transfer that information from deploying institutions to the manufacturer. They also need a process for analyzing it. Planning these arrangements during development helps establish whether the proposed monitoring can actually be carried out.

The discussion anticipated variation in how manufacturers would implement those plans. Some might establish dashboards and pipelines with deploying institutions; others could struggle to secure the necessary access. It also noted emerging tools that help hospitals track the AI models they use and help manufacturers support post-market surveillance and clinical follow-up.

A monitoring plan is only practical if the manufacturer can obtain the required data and use it to assess whether the product continues to perform as intended.

Account for Differences Across Regulatory Markets

Many of the development practices discussed in the webinar are relevant across the U.S., EU, and UK. However, the panel noted that expectations for data governance, training, testing, and performance demonstration could differ. New standards being developed to support the EU AI Act were expected to clarify how manufacturers should demonstrate compliance and how those expectations would compare with the U.S. approach.

The European discussion covered several related but distinct areas:

AI requirements: Data governance, transparency, training and testing data sets, and fairness were discussed in relation to the EU AI Act.
Individual rights: The panel connected European approaches such as the right to be forgotten to the history of data protection under GDPR.
Data access and portability: The EU Data Act was raised separately in relation to how information is handled and made available under particular conditions.

The panel also raised a privacy concern: advances in AI may make it easier to re-identify individuals in data previously considered adequately anonymized. That uncertainty adds another consideration to how organizations protect and use information.

The panel mentioned international efforts such as the International Medical Device Regulators Forum (IMDRF) to reduce differences across jurisdictions. However, it remained uncertain how closely European and U.S. expectations will align. The discussion suggested that the detail and rigor of the standards being developed to support the EU AI Act would help clarify the answer ot this question.

On preparing for EU AI requirements:

“They’re going to apply in some form, and if you want to put a device on the market here, you know, you should start considering those now.”

Sandy Wright, Head of Devices, Scarlet

Define the Scope and Controls for LLM-Enabled Products

LLM-enabled medical products are already reaching the market. The discussion cited certified ambient AI scribes while noting ongoing questions about how to qualify specific applications as medical devices in the EU and UK.

The more tasks an LLM-enabled product is intended to perform, the more situations the team needs to test to demonstrate that it behaves appropriately. Depending on a third-party LLM adds another challenge: a major part of the product’s functionality may come from software the manufacturer does not directly control.

Model governance therefore needs to address:

Which model and version the team has verified and validated
Whether that version can remain fixed, or pinned, rather than changing dynamically
What happens if the provider changes the model or stops offering the validated version
Which models and versions are available in each region
Where data must remain and where the model processes it

The panel described a patient-engagement feature that answers a patient’s question on the care team’s behalf when the answer exists in predefined material. Keeping that scope narrow reduces the range of behavior the team needs to evaluate. It also makes red teaming, or deliberately testing the system with challenging or adversarial inputs, more manageable.

That boundary can conflict with user expectations. Patients and care teams have access to general-purpose LLMs and may question why a product offers fewer capabilities. The development challenge is to balance that demand with the behavior the team can validate and the risks it can manage. Defining appropriate success metrics remains important for LLMs, particularly given the amount of validation work these products can require.

Expand Risk Management for LLMs and Agents

The panel described LLM and Agentic AI risk management as an evolving area. Alongside patient harm and delayed treatment, the discussion raised cybersecurity, data leakage, inappropriate chatbot behavior, and intellectual-property exposure. Teams developing agents also need to define what the agent can do and how they will monitor its behavior.

The discussion described separating AI capabilities within the medical-device function from those elsewhere in the connected software and services. Teams still need to understand and manage the risks in each setting, with particular care around behavior that affects the medical-device function.

Clearly defined language, organized data sets, and ontologies that describe concepts and their relationships can help teams guide AI outputs and assess whether those outputs match the information provided. Those resources also carry business value. Losing control of proprietary data or ontologies can expose intellectual property the organization invested in.

The webinar compared the development of LLM and agent infrastructure to early cloud computing, when teams had to build monitoring, failover, and resilience mechanisms that later became platform capabilities. Some AI controls may mature similarly, but that possibility does not resolve the risks of products being developed today.

Work on AI-tool validation and risk management in the software development lifecycle may also inform future medical-product practices. The panel described those techniques as developing faster in product-development workflows, with potential application to AI within medical devices. In both settings, teams still need to define scope, evaluate behavior, and manage the risks of the particular system they are building.

How Orthogonal Can Help

1. Develop AI Factory Processes and Data Strategy

Orthogonal helps clients establish processes for moving AI capabilities from discovery into development and deployment, including approaches that incorporate LLMs and Agentic AI.

2. Build SaMD and Connected Digital Ecosystems

Orthogonal develops Software as a Medical Device (SaMD) and the connected software and services around it, and helps organizations build their internal development capabilities.

3. Apply Modern Architecture and Development Practices

Orthogonal applies Agile methods and cloud architecture to medical-device software, including designs that separate components and support resilience when failures occur.

4. Automate Development Records and Design Controls

Orthogonal uses code-based approaches to automate development and design-control activities and generate records as the work proceeds.

5. Apply AI Tooling With Validation and Risk Management

Orthogonal applies AI tools to software development, testing, and automation, while developing practices to validate those tools and manage their risks.

Watch Webinar Recording

After the Webinar

Agentic AI Playbook: How MedTech Teams Must Evolve
The role-by-role framework behind the workflow shown in this session.
Automating Compliance with AI in the MedTech SDLC
How compliance evidence, testing and traceability become continuous outputs of development.
Agentic AI insights
All Orthogonal articles, webinars and playbooks on agentic AI in regulated MedTech.

Working on an AI-enabled device?

Orthogonal designs, builds and validates medical device software under design controls, from first prototype to post-market change. Explore AI-enabled medical device software development.

Bernhard Kappe orthogonal software as medical device

Randy Horton, VP of Solutions and Partnerships, Orthogonal