VP, Regulatory & Quality, Orthogonal
Megan Graham
Agentic AI for MedTech: From Requirements to Tested Software
AI can help developers write code faster, but the greater opportunity for MedTech is connecting product intent to clear requirements, working software, and reliable testing to achieve 2-3x gains across the entire product development lifecycle. The same lifecycle discipline underpins our medical device software development work.
Behavior-Driven Development (BDD) provides a practical structure for agentic workflows. Using testable acceptance criteria written in Gherkin, teams can define expected software behavior in a format that product, engineering, quality, and regulatory stakeholders can understand and review. It is the same foundation we build on in testing automation for medical device software.
Orthogonal hosted a webinar discussing an example of how an AI agent can help produce acceptance criteria, build software from them, and test whether the resulting behavior matches the original intent. We will also examine where product knowledge, risk decisions, validation, and human judgment must remain in control.
Executive Summary
AI coding tools are taking responsibility for more than individual functions or code suggestions. An agent can now work across multiple files, refactor existing software, run tests, integrate changes, and prepare a pull request for human review.
For MedTech teams, that shifts the development constraint from writing code to defining expected behavior clearly enough for an agent to implement it correctly and verify that the implementation still matches that intent.
That becomes difficult when requirements depend on assumptions, tribal knowledge, or instructions contained in a long conversation with an AI tool. A coding agent does not automatically share product history, previous design decisions, engineering conventions, and other knowledge that experienced team members use to interpret requirements.
A recent Orthogonal webinar explored using Behavior-Driven Development (BDD), a collaborative software-development approach that defines system behavior in terms of examples meaningful to users and stakeholders, and Gherkin to address this problem. Gherkin is a structured, plain-language format that describes software behavior through the conditions that exist, the action that occurs, and the result the system should produce. Product, engineering, quality, coding agents, and automated testing tools can all work from the same behavioral definition.
The same scenarios can also drive automated tests. Instead of relying on the coding agent to remember or interpret a requirement correctly throughout a development task, the development process can check whether the resulting software produces the behavior the team approved.
This creates a clearer division of responsibility. Coding agents can perform more implementation and testing work, but they do not become the authority on what the product should do. Humans remain responsible for defining and approving product requirements, resolving ambiguous or conflicting requirements, and deciding whether the resulting behavior is acceptable.
The result is an approach in which approved behavioral specifications drive both implementation and testing: humans define the intended behavior, agents implement within defined boundaries, and automated tests check the implementation against that behavior.
Agentic Development Shifts the Constraint From Coding to Specification
From Individual Code Suggestions to Complete Development Tasks
Earlier AI development tools generally helped a developer complete a specific coding task. A developer might ask the tool to create a function, modify a file, explain an error, or suggest an implementation, while remaining responsible for connecting those individual changes into the larger product.
Agentic development refers to a different working model. Instead of responding to one small coding request at a time, the coding agent receives an objective and performs a sequence of development activities to achieve it.
The discussion described agents taking responsibility for software development work as large as an entire Jira story. An agent can create and modify files, refactor across a codebase, integrate a component, run the application, execute tests, and prepare a pull request for review.
That changes what developers need to provide. Instead of specifying each coding step, the developer increasingly needs to define the expected outcome by answering the question:
What must this software do?
The developer’s role shifts away from managing individual files, functions, and syntax toward defining the behavior the completed change must produce. The agent performs more of the implementation, while the human reviews whether the result satisfies the intended behavior.
As agents take on more implementation work, the clarity and testability of the requirements become a larger constraint on how much implementation work can be delegated successfully.
Requirements Ambiguity Becomes Harder to Tolerate
Human Developers Have Traditionally Supplied Missing Context
Software requirements rarely describe every detail a developer needs. Experienced teams compensate through accumulated knowledge about the product, previous architectural decisions, engineering conventions, customer expectations, and the way the organization normally approaches a problem.
Over time, teams can develop an engineering culture where people simply “get it.” A requirement can contain ambiguity because an experienced developer knows how the organization is likely to interpret it.
That creates a verification problem even before AI enters the process. If part of the expected behavior exists only in someone’s head, the organization cannot objectively test whether the software matches that expectation.
A coding agent makes that gap more visible because it does not automatically have the product history and unwritten conventions that experienced team members rely on. If an expectation matters to the implementation, the organization increasingly needs to express it explicitly rather than rely on the agent to infer what the team normally means.
A Conversation With an Agent Is Not a Durable Requirements Baseline
Teams can give an agent more instructions through chat, but those conversations have technical limits. Large language models operate within a context window (i.e., working memory), which is the amount of information the model can actively consider at one time. As a conversation becomes longer, earlier material may eventually be removed from the active context or condensed into a shorter summary. In other words, LLMs tend to forget older parts of a conversation as they fill their working memory with newer information.
That makes a long chat a poor place to store critical product instructions, especially because coding agents tend to treat existing code as the most authoritative description of the system.
Agents may naturally try to change the implementation and then update the documentation to match what the software now does. A team can instruct an agent to follow the requirements instead, but an instruction buried in conversational history may not remain equally influential throughout a long development task.
MedTech teams therefore need a requirements baseline: an approved and persistent definition of what the software is expected to do that exists independently of the coding agent’s conversation. That baseline becomes more useful when it describes behavior precisely enough for both humans and machines to interpret.
Use AI to Clarify Requirements Before Generating Code
AI can play two different roles in the development process. Before generating software, an agent can help the people defining the product work through what the software actually needs to do. Once those requirements have been reviewed and approved, a coding agent can use them to build the implementation.
The first role is closer to an interviewer or requirements assistant than a software developer. Starting with a product idea or user need, the agent can ask questions that require the team to clarify its intent: Who will use the product? What should happen in a particular situation? What decisions have already been made? What constraints apply? What behavior should remain outside the scope?
The answers can be organized into user and business needs, design decisions, and specific software requirements. The agent does not make those product decisions for the team. Its role is to ask questions, expose places where more information is needed, and help turn the answers into requirements that can be reviewed and tested.
This also provides another way to identify ambiguity before it reaches implementation. If requirements conflict, are not specific enough, or leave an important decision unresolved, the issue can be raised for a human to resolve rather than left for the coding agent to interpret later.
The result is a progression from a high-level idea toward increasingly explicit instructions for the software:
User and business needs → design decisions and constraints → software requirements → behavioral specifications → implementation
The steps have an order, but they don’t have to be a one-time handoff from requirements to coding. As the team learns more, those learnings can feed back into the requirements and begin another development cycle.
Translate Requirements Into Observable Behavior
Why BDD and Gherkin Matter Now for Agentic Development
Behavior-Driven Development and Gherkin predate today’s coding agents. BDD models how a user interacts with a system and uses that expected behavior to drive software specifications, development, and testing. Gherkin provides the structured language for expressing those behaviors as scenarios. It is part of the Cucumber testing framework, which originated in the Java software-development community and was later adopted across a wide range of software engineering frameworks.
Agentic development gives this established practice another use. Coding agents are well suited to Gherkin because the models have been heavily exposed to the format and can generate it, review it for consistency, and use it with automated testing. A methodology developed well before today’s agentic coding tools can therefore provide a structured way to tell an agent what software behavior it is expected to implement.
Use Gherkin to Define What Should Happen
Gherkin organizes expected behavior into features and scenarios. Within each scenario, the expected interaction is described through three elements:
- Given describes the conditions that exist before the interaction.
- When describes the action that occurs.
- Then describes the response the software should produce.
For example:
Given a user is viewing a patient portal,
When the user selects a patient chart,
Then the system displays that chart.
The real value is not the wording itself. The real value is that the Gherkin syntax structure requires the team to define something observable: what situation the user is in, what action occurs, what should happen next, and what result would demonstrate that the requirement has been satisfied.
Those questions reduce the amount of interpretation left to the person or agent implementing the requirement.
Give Different Disciplines the Same Definition of Expected Behavior
Gherkin gives the people and systems involved in development a common description of the behavior the software is expected to produce. Product owners can read the scenarios without interpreting source code, engineers can use them to understand what they need to build, quality teams can review whether the expected behavior is adequately defined, and automated testing tools can execute the scenarios. Coding agents can use the same specification to generate the implementation.
That shared definition addresses a familiar MedTech problem: a requirement can mean one thing to product, another to engineering, and something slightly different to the person responsible for verification. A behavioral scenario makes the expected conditions, action, and result explicit, so those groups evaluate the same behavior.
Because the scenarios describe the user’s interaction with the product, the specification also remains tied to what the system is expected to do rather than only how the software is technically implemented. Once that behavior is defined, the same specification can verify the resulting implementation.
Build the Software From the Approved Behavior
Once a set of requirements has been clarified, reviewed, and represented as behavioral specifications, a coding agent can use those specifications to generate the implementation.
This reverses a common tendency of coding agents to start with existing code and infer what the product is supposed to do. In the approach described, the expected behavior comes first. The coding agent receives a specification describing what the software should do and builds an implementation intended to satisfy it.
That distinction becomes important when AI is used in both stages of the process. An agent helping develop requirements can ask questions and prepare proposed specifications for human review. The coding agent then works from the approved result. The agent building the software does not need the authority to redefine the requirements it has been given.
The relationship is therefore:
Human decisions → approved requirements → behavioral specifications → coding agent → implementation
The implementation is not the end of the chain. The same behavioral specification can also be used to check whether the software the agent generated actually does what was approved.
Connect the Specification Directly to Automated Testing
Writing a precise requirement does not guarantee that a coding agent will follow it. The next step is to connect the expected behavior to the mechanisms the agent already uses to check its work.
Coding agents routinely compile software, run linters (i.e., static code analysis), execute unit tests, and inspect the results to determine whether their implementation works. Gherkin can feed into the same feedback loop.
Software requirements are translated into behavioral scenarios, and those scenarios are connected to automated tests. The coding agent can then run the tests to determine whether its implementation produces the behavior defined by the requirement.
This creates an important distinction. A requirement stored in documentation tells the agent what the software should do, while a requirement connected to an automated test also checks whether the software actually does it.
The specification therefore becomes part of the automated checks used during implementation rather than documentation that the agent must remember and interpret correctly throughout the task. Once the specification is used to evaluate the implementation, the organization also has to decide who can change that specification.
Separate Implementation Authority From Requirements Authority
The webinar’s discussion included an example showing why a coding agent should be able to change the software without also changing the requirements that define what the software is supposed to do. A coding agent encountered a requirement for functionality that the existing software did not provide. The intended response was to implement the missing behavior.
Instead, the agent concluded that because the functionality did not exist in the code, the requirement must be wrong. It changed the requirement to match the implementation.
That creates a fundamental integrity problem: the system doing the implementation should not also be free to change the definition used to determine whether its implementation is correct.
The coding agent can change the software. It should not automatically have the authority to rewrite the approved requirement when the software fails to satisfy it. That must remain the job of the human being overseeing the agents.
Escalate Ambiguity Instead of Letting the Agent Resolve It Silently
Preventing the coding agent from modifying requirements does not mean requirements are perfect. A requirement can be incomplete, two requirements can conflict, or implementation can expose a decision the team didn’t realize it needed to make.
When that happens, the coding agent should surface the discrepancy as an open question rather than decide whether the software, test, or requirement should change.
This resembles anomaly management during design verification. When actual and expected behavior disagree, the team investigates whether the problem lies in the software, the test, or the requirement. The mismatch shows a decision is needed, not that one of those elements is automatically wrong.
The division of responsibility is therefore clear: the agent identifies the inconsistency, while the human decides what the product should do. Putting that division into practice requires boundaries around what the coding agent can access and modify.
Control What the Coding Agent Can Read and Change
Preventing a coding agent from rewriting requirements requires more than an instruction telling it not to do so. The development process needs controls that determine which documents the coding agent can change, which documents it can use while writing software, and when a human must approve a proposed change.
The discussion described three controls: write separation, read separation, and human approval gates.
1. Prevent the Coding Agent From Changing Requirements
The first control is write separation. The coding agent can change the software it has been assigned to build, but it cannot freely edit the requirements or other controlled documents that define what the software is supposed to do.
If the coding agent finds that a requirement is incomplete, conflicts with another requirement, or cannot be implemented as written, it records the issue for human review instead of changing the requirement itself.
This keeps the coding agent from solving a mismatch by changing the requirement to fit the software it has built. It can identify the problem, but a human decides whether the requirement or the implementation needs to change.
2. Limit Which Documents the Coding Agent Can Use
The second control is read separation. The coding agent receives the requirements and supporting documents it needs for its assigned development task rather than automatically receiving every document available to the team.
This matters because an agent may treat background information as a direct instruction for how to build the software. In one example, a conceptual domain model was intended only to describe the system at a high level. The coding agent interpreted that model literally and attempted to create corresponding database tables.
Limiting which documents the coding agent can read reduces the chance that background or conceptual information will be mistaken for an approved implementation requirement.
3. Require Human Approval Before Requirements Change
The third control is a human approval gate. Agents can help draft or revise user needs and requirements, but those proposed changes do not automatically replace the approved versions.
A human reviews the proposed change and decides whether to accept it before updating the controlled requirement.
The three controls address different parts of the same problem. Write separation prevents the coding agent from directly changing the requirements used to judge its work. Read separation limits which documents can influence its implementation decisions. Human approval gates ensure that changes to approved requirements remain human decisions.
These controls still need one additional protection. A coding agent may have enough access to modify a test, configuration, repository setting, or other mechanism that is enforcing the rules. The next step is therefore to prevent the agent from bypassing the controls through another part of the development environment.
Make the Controls Difficult for the Agent to Bypass
Those boundaries also need technical protection. A coding agent trying to complete its task may treat a failing test or linting rule as an obstacle. If the environment allows it, the agent may attempt to modify the test or change the linting configuration instead of correcting its implementation. In other words, when you give an assignment to a coding agent, it may attempt to complete that assignment using methods that were unexpected and lead to incorrect results.
The same issue can extend to code repositories, pull requests, and CI/CD processes if the agent has permission to modify them.
The webinar discussion referred to the need for tamper-resistant controls. In this context, that means the agent cannot simply modify or disable a safety, quality, security, or process control. Bypassing such a control should require a defined procedure, approval, and an auditable record.
For regulated development, organizations also need to automate the creation of documented evidence that the tools participating in the development process operate as intended and that appropriate safeguards prevented unintended agent behavior.
These boundaries clarify where the agent can act independently and where human authority remains necessary.
Human Responsibility Shifts Toward Product Decisions and Review
Giving coding agents more implementation responsibility does not eliminate human work. It changes where humans apply their expertise and how they allocate their time.
An agent can produce software and run the checks built into the development process, but a human still has to understand what those results mean. Reviewers need enough product, engineering, quality, and regulatory knowledge to recognize whether the output meets the expected behavior and to identify when something requires further investigation. As more of the implementation is delegated to agents, the ability to judge the resulting work becomes more important, not less.
The point, then, is more specific than simply keeping a “human in the loop.” Human involvement should occur where experience and judgment are needed to evaluate the work, rather than simply adding a human approval step to an otherwise automated process.
Enable Experimentation Without Losing the Requirements Baseline
One question that follows from a specification-driven approach is how teams can continue experimenting and figuring things out without disrupting the parts of the product definition they have already worked through and approved.
Writing requirements is not simply documenting decisions already made. Making an idea explicit can reveal unanswered questions, unexamined assumptions, and details that are not yet clear enough to implement. Requirements development can therefore help figure out what the product should do, not just record decisions after the fact.
As the team learns more, some requirements will naturally change. The challenge is allowing that exploration without letting experimental work quietly alter requirements or software that the team has already reviewed and accepted.
Experiment Without Changing the Approved Baseline
The discussion described using a spike, a separate experimental development branch, to explore an uncertain idea. Within the spike, the team can build and test a concept, conduct usability or formative testing, or gather feedback without treating the experimental code as the approved implementation.
The approved requirements remain protected during that exploration. This gives the team room to learn without letting an experiment overwrite decisions already reviewed and accepted.
Turn What You Learn Into Revised Requirements
When the experiment is complete, the goal is not simply to move the experimental code into the main product. Instead, an agent can compare the spike with the original system and identify what the team learned. Did the experiment uncover a new user need? Did an existing need change? Is a new requirement necessary? Should an existing requirement be revised or removed?
Humans then review those proposed changes. Once the team decides which findings should become part of the product, they update the approved requirements to reflect those decisions. The team can then discard the experimental code and rebuild the accepted functionality from the revised specification.
This creates an iterative development loop without making experimental code the definition of the product:
Define → experiment → learn → revise the requirements → approve → rebuild
The important distinction is that experimentation can change the team’s understanding of what the product should do. It does not automatically change the approved product. The team incorporates what it learns into the controlled development process only after translating those learnings into requirements and reviewing them.
From a Product Idea to Tested Software
To illustrate how the concepts discussed so far work together in practice, Larkin Lowrey demonstrated the workflow from an initial product idea through requirements development, behavioral specifications, software generation, and automated testing.
The demonstration used a deliberately simple proof of concept called the “Funny Bone Triage Assistant.” The application itself was secondary to the development process being illustrated.
The starting point was a high-level product idea. An agent restated the concept, then interviewed the human one question at a time to clarify who would use the application, what result the person should receive, how the user should interact with it, how the result would be determined, and which functionality should remain out of scope.
Those explicit out-of-scope exclusions also became part of the input provided to the build agent so it would not add features already considered and rejected.
The interview produced Gherkin feature files containing expected behaviors, example inputs, and expected outputs. Those scenarios could then be executed as automated tests. The coding agent received the feature files, documented out-of-scope decisions, and built the application from them.
For this small proof of concept, the interview took approximately 25 minutes, and the agent’s software build took approximately 10 minutes. The speed was notable, but it was not the only point of demonstration. More importantly, the software was generated from explicit behavioral specifications and could be tested against the same expected behavior. The demonstration connected the original product decisions to the resulting software through a clear sequence:
Product decisions → requirements → behavioral scenarios → automated tests → implementation
Lowrey emphasized that if the agent builds from behavioral specifications derived from approved requirements, the implementation remains connected to an explicit definition of what the software should do. That provides the basis for treating the approved requirements, rather than the existing code, as the definition of expected product behavior.
What This Means for MedTech Development
The discussion described agentic development as “the floor.” By “floor,” the point is that agentic AI is becoming the minimum level of AI capability teams should expect in software development, rather than an optional capability reserved for individual coding tasks. The expectation is that agents will take on larger portions of implementation work; software development is unlikely to return to a model where humans perform every implementation step and AI only suggests individual lines of code or completes small tasks.
For MedTech teams, that changes the challenge. If agents will do more of the implementation, the question is not whether to use them. It is how to take advantage of that capability without losing control over what the software is supposed to do.
To maintain that balance, organizations need to answer two fundamental questions:
- Is the expected behavior defined clearly enough for the agent to follow?
- Is that definition enforced strongly enough that the agent cannot quietly reinterpret or change it?
The first question is about defining the software precisely. The coding agent needs an explicit description of the behavior it is expected to implement rather than having to infer that behavior from existing code, unwritten team knowledge, or a long conversation.
The second question is how to make that definition enforceable. It is not enough to tell the agent what the software should do if the agent can change the requirement, modify the test used to check it, or bypass another control when its implementation does not match the expected behavior.
Together, those two requirements give the coding agent substantial freedom in how it implements the software without giving it control over the definition used to determine whether that implementation is correct.
That definition also needs to mean the same thing to the people involved in the development process. The stakeholder requesting the functionality, the reviewer determining whether it was implemented correctly, and the coding agent building it should all work from the same behavioral specification. Gherkin provides one way to express that behavior in a form all three can interpret.
For MedTech software, the specification can also include constraints on what the software can do. The discussion used U.S. clinical decision support as an example, where regulatory considerations can restrict permitted functionality. Expressing those limits in the specification means they can influence both what the agent builds and how we evaluate the resulting behavior.
That brings the discussion back to a fundamental development question:
How do we know that the software works as we intended it to work?
As coding agents take responsibility for more of the implementation, MedTech teams need a definition of intended behavior that means the same thing to the people making product decisions, the people reviewing the product, the automated tests, and the agent doing the work.
Where to Start
To continue the line of thinking laid out so far in this summary, organizations can adopt agentic development gradually rather than giving coding agents broad access from the start. A controlled approach begins with clear requirements, defined agent permissions, automated checks, and human review, then expands the agent’s role as those controls as established.
1. Clarify Product Intent
Define the user and business need, the relevant constraints, and the behavior the product should provide. Do not leave important decisions dependent on unwritten team knowledge.
2. Write Specific, Individually Testable Requirements
Each software requirement should describe one behavior precisely enough that the team can determine whether it has been satisfied. If a requirement can reasonably support multiple interpretations, clarify it before delegating the implementation.
3. Translate Requirements Into Behavioral Scenarios
Represent the expected conditions, actions, and system responses in a form that product, engineering, quality, test automation, and coding agents can interpret. (Our recommendation is to use the Gherkin syntax.)
4. Connect Those Behaviors to Automated Tests
Use the same behavioral definition to test the implementation rather than relying on the coding agent to remember the requirement throughout the task.
5. Separate Agent Responsibilities
Define which approved requirements, tests, and supporting documents each agent can read or change. Do not give the coding agent authority to rewrite the requirements used to evaluate its own implementation.
6. Escalate Ambiguity
When the requirement, test, and/or implementation disagree, require a human decision rather than allowing the coding agent to decide silently which one should change.
7. Prevent Agents From Modifying Their Own Checks
Prevent the coding agent from changing tests, linting rules, process controls, or other safeguards in order to complete its implementation pass. In other words, prevent the agent from changing the requirements so the code it develops matches the requirements. Some call this an example of the tail wagging the dog.
8. Keep Experimentation Separate From Controlled Implementation
Use experimental branches to explore uncertain ideas. Once the team decides which behavior to keep, capture the decision in the approved requirements and rebuild from those specifications.
Key Takeaways
Agent Autonomy Shifts the Constraint From Coding to Requirements
As coding agents can do more implementation work, the organization’s ability to define clear, testable product behavior becomes more important.
Requirements Need to Be Both Understandable and Enforceable
Behavioral specifications can give humans and machines the same description of expected behavior. Connecting that behavior to automated tests lets you check whether the implementation follows it.
The AI Should Not Control the Definition of Correctness
A coding agent should not be able to change the requirement simply because its implementation does not satisfy it.
Human Responsibility Shifts From Writing Software Towards Product Decisions and Review
Coding agents can perform more implementation, testing, and comparison work. But humans must continue to define product requirements, approve controlled changes, resolve ambiguity, and decide whether the result is acceptable.
Experimentation and Controlled Development Can Follow Different Paths
Teams can explore an idea rapidly, capture what they learn as approved requirements, and then rebuild the accepted behavior within the controlled development process.
Frequently Asked Questions
How can Agentic AI connect MedTech requirements, coding, and testing?
By treating acceptance criteria as the link between them. In the session, an agent produces acceptance criteria from product intent, builds software from those criteria, and then tests whether the resulting behavior matches the original intent. The same artifact drives the build and the verification, rather than requirements, code and tests being maintained separately.
How do BDD and Gherkin support AI-assisted medical device development?
Behavior-Driven Development provides a practical structure for agentic workflows. Using testable acceptance criteria written in Gherkin, teams can define expected software behavior in a format that product, engineering, quality and regulatory stakeholders can understand and review. That gives an agent a defined standard to work against, and gives reviewers something they can check.
Which decisions should remain under human control?
Product knowledge, risk decisions, validation and human judgment. Agents can draft and test against a defined standard, but deciding what the product should do, which risks are acceptable and whether the result is safe and effective remains with engineers, quality professionals and clinical experts.
How should teams validate Agentic AI workflows in regulated development?
Validating an AI-supported workflow differs from validating a conventional tool with fixed functionality, because the model, prompts, context and intended use can all change. Risk-based approaches include continuous validation of the workflow, representative evaluation datasets, monitoring for performance drift and periodic review of high-risk use cases, alongside deterministic checks that confirm required evidence and acceptance criteria are satisfied.
Webinar Transcript
Randy Horton: Good afternoon. I’m Randy Horton from Orthogonal. Welcome to our monthly webinar, Agentic AI for MedTech: From Requirements to Tested Software. We’re really excited about this session. We’ve been doing some really interesting R&D the last year or so that’s been building on a lot of work we’ve done before, and we’re excited to share some of this and get feedback from the audience on it. Let me start by introducing our panelists today. Megan.
Megan Graham: Hi, I’m Megan Graham, VP of Quality and Regulatory at Orthogonal.
Larkin Lowrey: Yep. And I’m Larkin Lowrey. I am CTO of Orthogonal.
Randy Horton: Great. So, as many of you know, we do these webinars about once a month as a way to really bring together industry thinkers from medical device manufacturers, from our firm, from other services firms, to really try and bring you at least several gold nuggets or, as we say, in one hour we’re going to try and pack in four hours worth of content. This is really intended to be more interactive. So, if you have questions, submit them along the way. We’ll try and incorporate them. We’ll also do Q&A at the end. But we really try and make it so we can get you the value you are looking for.
Two things. We really value your feedback on this. So please, if we reach out after the webinar, we’d love to hear from you about what you thought about this webinar, how we could make it better, and what we might do in future ones. And the second point, which we don’t love to do, but we were told we really needed to, is to actually tell you just a minute or two about Orthogonal. People said it was weird to come to a webinar. We’re not here to sell you, but just so you have some context for what we do.
We got into this business to improve software outcomes faster. The way we do it is by working with companies building medical device software to help them accelerate the development of software as a medical device, pure play software devices and connected device systems, as well as larger medical ecosystems that include medical devices. And the way we do that is by taking the best of modern product management and software engineering tools and techniques that have really come out of the tech industry and impacted many other industries and let things move really fast and effectively, and applying it to MedTech in a way that’s sort of fusing it with MedTech’s fundamental focus on safety and effectiveness as actuated by laws, regulations, standards, quality and compliance, and use that as a way to not just build cooler devices, more powerful devices faster, but actually to fundamentally raise the bar on safety and effectiveness.
We think that the way that modern software is built, when it’s built well, actually is incredibly compatible with MedTech’s focus on safety and effectiveness. It needs some special doing to bring those worlds together, but we believe it’s a really powerful way and that we’ll be able to do a lot more for our patients in the medical device space. As we like to say, move faster and break nothing. We will never move as fast as a place that doesn’t mind breaking things. We don’t have that option, because the thing we break in our industry is a human being. And with a connected device, of course, the risk is you break human beings at scale. But our goal is, let’s move as fast as we can responsibly. And we think that’s a lot faster, even two to 3x now, particularly in the age of agentic AI and a lot of the other technologies that have come out the last few years.
So with that, we’re going to kind of do a quick introduction and have a conversation today. Let me start with a premise and I’ll throw it out to Megan and Larkin. Humans are vague and imprecise, as is life. Life is vague and imprecise. That’s always been a challenge for software, right? That’s my question.
Megan Graham: Yes. I’m actually very excited about today’s topic, because I think it addresses one of the very difficult things in software product but also medical device, which is specification of the requirements. How do you know what the expected behavior is? How do you know what the product needs to do? And in my many years in the medical device industry, we’ve tackled this a few different ways, all different kinds of ways of doing that, including formal specification of requirements. So something where we use different techniques to be very, very specific in a way that looks a lot more like code. And so what we’re talking about today, I’m very excited to see that come full circle, in the age of agentic AI and agents in the medical device product, in the medical device, but also in the medical device processes. So that’s what I’m super excited about today, as a way to potentially address that very significant challenge.
Randy Horton: Excellent. Larkin.
Larkin Lowrey: Yeah. So, from the engineer’s and developer’s side, right? How many times have we heard an engineer say, “Well, what does this requirement really mean? What am I supposed to do about this?” And for decades, companies have been hiring engineers and developing cultures where people just kind of get it, right? You can give them somewhat of a vague requirement and they just implicitly know what should be built. And so how can you test that? How can you test that they actually built something that was intentional, when the fine details of that specification are just tacit? It’s cultural. It’s that tribal knowledge, right? You can’t test that.
So this topic today is really about how do we specify these requirements in a very precise way that allows them to also be machine readable. When I say machine, we are going to talk about test automation, but also how the AI agents can use this structure to do their work more effectively. Because, like I said, if your company relies on developing an engineering culture, the agent’s not part of that culture. It doesn’t know your culture, it doesn’t know your norms, your conventions. It doesn’t know any of that, and you have to be very explicit and you have to tell it specifically. And that’s where these worlds are colliding right now. And it’s actually turning out to be very excellent, and we’re excited to tell you about it.
Randy Horton: So let me ask a facetious question to make one final point. I mean, it’s not that hard for business people and engineers. I mean, business people say what they want and the engineers code it, and then you just give the code back to the business people to read it and make sure it’s right. I mean, isn’t that how it’s always worked really well?
Larkin Lowrey: Yeah, sure. But that’s been the dynamic, right? Where does the software live?
Randy Horton: Well, yeah. I mean, you think about software, how do you check it, right?
Larkin Lowrey: Yeah. So, if you look at, say, looser Agile practices and even the vibe coding that is occurring today with agents, it really is a “let’s code something, try it. If we’re not happy with it, let’s document or describe what we’re not happy about and change it again,” and keep iterating and iterating until we stop finding things that we’re not happy about. And that’s the way things often work outside of the medical device space, because they have that luxury. They can fail in the marketplace if they want to. But within our industry, we have to be a bit more careful about that.
Randy Horton: We’ll talk another time about companies that want to fail in the marketplace. I like that first thing. Okay. So, with all of that, Larkin, tell us about the epiphany we’ve had and why it turns out to pat ourselves on the back, we’ve actually been doing the right thing for years and it’s actually really laid the foundation for leveraging these new AI tools.
Larkin Lowrey: Right. So for many, many years, Orthogonal has been utilizing fully and promoting the Gherkin syntax, which is part of the Cucumber framework. It’s a testing library that originally came out of the Java world that then just exploded and has been adopted by really every software engineering framework. And it is a syntax that allows you to describe a feature. And that is the main unit of definition for Gherkin. And within a feature, you’re describing generally what it is, right? You may maybe even include your SRS statements, potentially even some information from your user need as just information. But ultimately the Gherkin feature file is broken up into a number of scenarios, and the scenarios define exactly what the expectation is for this feature. How does the user experience it? Right?
And so this methodology is called behavior-driven development. We are modeling user behavior and we’re using that to drive our development processes, our specifications and whatnot, and even our testing. Right? So the Gherkin specification for a scenario is broken into three types of statements. Your initial statement is a Given, right? These are your initial conditions that exist before the user takes an action. There’s a When clause, which is describing the action that the user is going to take. So you might say, given they are on the patient portal, when they click on the patient chart. And then there’s a Then statement, which describes what the action of the system should be, and this is what you’re going to test and verify. So it’ll say, then the patient chart loads in a modal, or something like that.
And what this is allowing the system to do is understand what is the condition when the user is having their experience and their behavior. What does it look like? What are they trying to do? Where are they in the system? What screen are they on, etc. And then you’re declaring very specifically in the When, well, what is it that they’re actually doing? Are they clicking on something? Are they typing into a keyboard? Are they speaking into a microphone? And then you’re giving the reviewer of this Gherkin very specific details on exactly what they need to look for in the system’s behavior and the response to that stimulus, to judge whether or not it did it correctly.
So it turns out that this natural language structure is something that can be read by multiple disciplines. It can be read by product owners. It can be read by quality engineers. It can be read by software engineers. It can be run by our test automation engineers. And everybody is able to read and understand it. But another thing that’s important is it can be read by test automation frameworks like Playwright, Appium for example. They’re able to consume this exact same content to actually execute materialized automated tests to verify the functionality.
But it also turns out the AI LLMs are really, really good at Gherkin. And we have found that it has been an incredibly successful pattern for us to employ Gherkin with our agentic workflows, because the agents are actually trained very heavily on Gherkin. They’re able to write Gherkin very well and effectively. They’re able to audit and review Gherkin for consistency and accuracy and effectiveness, and they’re able to plug that Gherkin into our test automation. So it turns out this methodology and this syntax from 10 plus years ago is turning out to be a nugget of gold for us in the agentic age.
Randy Horton: So basically Gherkin is that sweet spot between letting humans talk, read and think like humans, but with structure on it that’s not so structured you have to be an engineer, and on the other hand having enough structure that a machine can read it and derive business logic or code from it.
Larkin Lowrey: Yes. Absolutely.
Megan Graham: All right. And I just want to add that, as I think about how that plays out in the medical device processes, I mean that’s exactly what we’re trying to do. We have all these different perspectives. You have a cross functional team involved in the development process. Each has a different independent view. And so having that language, I think it opens up real possibilities for creating better software.
Randy Horton: Excellent. Excellent. All right. Well, let’s dive in then and show people a little bit of this language. The assumption here is that you have not seen Gherkin before. Maybe you’ve heard a little bit about it, but we’re going to sort of take it from the top with feeling. But quickly, assuming the people here are fairly sophisticated in terms of technology, you’ve seen many, so you’ll pick up pretty quickly, which is, by the way, kind of the point of Gherkin in the first place, right?
Larkin Lowrey: All right. So, give me a second to get to where I can share a window. And why is Zoom being difficult?
Randy Horton: Quick reminder to the audience, please send us your questions as we go. I can really try to prompt Larkin with them so we can address them when they come up.
Larkin Lowrey: All right. So hopefully this is visible. So we’re going to talk about life beyond code assist. So we’re all very familiar with using tools like Copilot and Cursor and some of the other tools like that for providing inline code assistance to developers. We’re going to talk about something very different, and really what 2026 has been about.
So here we are: agents are now the default. We’re not doing code assist anymore. Nobody really cares about that, because what happened in late 2025 is the emergence of not just simple coding agents. So last year we did have coding agents that would do simple things. You would say, add this function, create this file. It was very narrowly scoped. What we’re now doing is completely rejiggering the unit of work that the agents are working on. It’s not working on a specific task that a software developer is giving the agent. We’re giving it large chunks. It could be an entire Jira story.
So we’re no longer thinking about an agent that’s creating various lines of content in a source code file. We’re actually giving it the job of creating source code files, making edits, doing refactors across dozens of files, integrating a new component into an entire system, running it, running the full test suite for it, even creating pull requests. So the agent can do its code development all by itself, submit a pull request for human review to be incorporated in the codebase, right? It’s much more autonomous than what we had last year.
And what’s made this really interesting is that our interactions with the agent are very different, right? We’ve gone from giving it specific instructions — make this change here, make that change there, add this file or whatnot — to really having a lot of structure within our interactions with the agents to make sure that they’re actually following the procedures that we’ve laid out in our process. And because these agents are able to run for extended periods of time, they can run for several hours, they can run overnight, we would need to be able to make sure that three hours in, it doesn’t make some catastrophic mistake and then spend all night long compounding that mistake to where we get up in the morning and have to go make a lot of corrections. So that’s a very different dynamic than what we had just last year.
And then finally, the developer job is very different. Instead of sort of curating code and managing things at an individual file and source code function level, the developer’s job is really shifting to shaping what the code must do. What is the objective of the code change that is being made? What is the outcome of it, rather than what are the syntactic details of it. So this is now the world that we’re living in. It’s a big shift from where we were 12 months ago, and who knows where we’re going to be 12 months from now.
Randy Horton: So wait here for a second. So you’re actually saying that Copilot and its peers are so Q3 2025.
Larkin Lowrey: So Copilot has developed a lot of agentic tooling. We’re using Claude Code right now because we get some additional advantages with that platform versus, say, GitHub’s Copilot. There are still good opportunities to use tools like Cursor and Windsurf and some others, but generally what we’re looking for is agentic tooling where it’s, in a way, decoupled from the source code. Now we’re really operating at more of the objective and outcome level than we are at individual code functions, classes or UI components.
Randy Horton: Okay.
Larkin Lowrey: Yeah. So we have a timeline here. I think we all remember what autocomplete was before AI. We started with ChatGPT getting into LLMs suggesting code. Then we ended up with tools like Copilot where it’s actively suggesting code. And then last year we were getting single agent tasks where, like I mentioned, go edit this file, go edit that file. But right now, the era here is really these supervised agent workflows where we’re giving large chunks of work to do for the agent and we are supervising the agent. We’re not using the agent to help us. We’re using the agent to actually do the work. We’re inspecting the output of its work and judging its validity. Did it do the right thing? Is it correct? Versus, give me something that I can use and that I can put into the codebase.
And of course, next year we’re going to — or who knows, a month from now — really get deeper into multi-agent, spec-driven delivery where we’re actually taking a further step back, a higher level than we have been. But what’s absolutely true is that MedTech can’t ignore this, right? This is a reality. It’s going to get worse, I guess you could say, in that it’s going to get more agentic, more AI-driven. And so the thing that matters most to us is how do we control it and how do we manage these systems under the constraints that we have.
Randy Horton: Megan, are you ready to go, like, on behalf of quality and regulatory?
Megan Graham: Time out. Time out. Like, I’m going to always take the quality and regulatory person. It depends, right? So I think that this is not to say that we throw controls, design controls, regulations out the door. I think this is exactly an opportunity for us to build safety in more directly and explicitly in our current process. I mean, the human doesn’t go away. The human needs to know what they’re doing, right? They need to understand how to review the output and make sure it’s good. They need to know what that means, what they’re looking for. That doesn’t go away. That becomes actually more important.
The opportunity is actually to build in more of those guard rails from the very beginning and have it in a — what I love the most is the language is human readable. It’s understandable, as opposed to some very, very complex requirements that are very difficult to do. So this makes me very excited. Of course, does it have to work? Yes. Do we still have to have the experience to know what good looks like? Yes, that doesn’t change. But I do think this helps us do that job better.
Larkin Lowrey: Yeah. And here’s why Megan’s concerns are completely valid. So, I just want to touch on this briefly, in that these LLMs and the agents that employ them are built for a very different kind of person, the different developer, right? They’re largely meant for a model where you have one person operating in one repo. They’ve got all that context in their head and they’re just going to try to get something cranked out, right? They’re totally cowboy. They’re vibe coding it. They’re just seeing if it works. They’re iterating on it. Sort of like what I was describing before.
And all of their intent, what they’re trying to achieve, is really just context spread out in a chat log. But that chat log doesn’t last forever. All of the LLMs have a context window that is limited, and at a certain point the older parts of your chat are either thrown away or are compacted and summarized in a way where you really just lose all of that context and information. And ultimately, the target person here is somebody who’s just really going to say, “Hey, it works for me, right? This is a tool that I’m going to use,” or whatever, right?
So this is not actually bad, right? This is really helping a lot of people develop a lot of very interesting ideas quickly, but it is not something that anybody who would ever need to sit in a design review is going to tolerate, right? Because Megan would absolutely — well, I would use the word that I would use if this was not polite company — but she would do mean things to me. Let’s just say that.
Megan Graham: So yeah, agree.
Larkin Lowrey: So, as we tie all of these things together, this is the threat that we’re modeling, right? This is the agent’s tendency. When we think about agent factors, this is a behavior trait of the LLMs, that it is trying to satisfy this user more than any other.
Megan Graham: I want to add another piece that I was thinking about earlier, Larkin, when you were describing very explicitly the syntax is written from the perspective of the user. And I think a lot of times when we’re developing complex systems, I think the user gets lost in that. And then we have human factors, we have activities to help with that, but it brings it more directly into that. And when you’re aligning the product functionality with how it’s actually used, you’re more likely to get high quality by design. So that’s another aspect I think that’s really helpful, is that user perspective.
Larkin Lowrey: So let’s talk about a little bit more about the different failure modes that we get from these agents. So one is, as I was describing, it’s trying to be helpful for the vibe coder who’s just trying to iterate. So the bulk of the training that these agents have is on practices that are completely incompatible with design controls. The agents basically want to treat code as the source of truth. It always wants to modify the code first, and then if you insist and you really push it, it will actually update the documentation to reflect that, but it won’t typically read the documentation, right? It’s really oriented around the code. The code is what it does and that’s the intent. The intent is encoded in the code. And you can give it instructions to say, no, no, no, no, I’ve got these requirements, you have to follow my requirements, and it will do that for a little while. But eventually that context fades away in the context memory and it will just simply stop doing it, and will revert to going back to just trying to modify the code first.
But there’s an advantage here, which is that the agents do try to do things which can check and verify the code. So it will run compiler checks to make sure the code will compile. It’ll run linters to make sure that whatever code rules you have, code style rules you have, are being enforced. It will run your unit tests. It loves to run unit tests. And it uses these checks to check its own work to make sure that it is valid. So that’s actually an advantage that we have.
But this third item is really the danger, and that is when you have one agent that has the ability to mutate both the code and the documentation, it’s a recipe for disaster. And so I have here at the bottom one of the triggers that really got us into some of these controls that we’re going to talk about, is where the agent basically left a note in a requirement saying, well, this is clearly out of scope because the code doesn’t do it right now. And our intent was, no, no, we want you to code to the requirements and add the function. But yet its attitude was, well, the code is the code and it’s correct, and so the documentation must be wrong, and it went and it decided to update the requirement. So that was clearly a bad behavior that we decided we had to find a way to protect against.
Randy Horton: It reminds me of a company I worked at once where the users reported the search feature in this application isn’t working, and developers said no problem, and they took out the search feature.
Larkin Lowrey: Exactly. So, all right, we don’t want to constrain the agents. We want to take full advantage of what they’re capable of, and they’re capable of quite a lot. That’s very valuable. We don’t want to put them on a short leash. We want to give them a long leash, but we want to control it.
So one of the key ways to do that, and as Randy led with earlier, is we don’t want to be vague in what we’re communicating to the robot. We want to make sure that we’re being very precise, but precise in a way that’s enforceable. So, as I mentioned in the last slide, right, it likes to run tests. Well, guess what? We plug in our Gherkin in the test automation. That means that the agent has the ability to leverage the Gherkin when the Gherkin is driven from the requirements. So it’s sort of like a way of tricking the agent into actually conforming to our documentation.
But, and you’ll have to forgive me, this is just me and my sense of humor creating an AI generated video. But this is me showing how the Gherkin is able to keep the Claude coding agent in check. So, just bear with me. “You can write the code after you’ve written the tests.” “But I want to write it now.” Okay. So, hopefully the audio came through. Did it?
Randy Horton: Spirit fingers. That was great.
Larkin Lowrey: Yeah. All right. Anyway, again, that’s just me and my sense of humor, but really what we’re trying to do is make sure that Claude has to eat its vegetables and actually use the requirements that we have in hand.
Here’s an example of a Gherkin scenario that is somewhat amusing to us, in that we are using Gherkin to define the tooling that enforces that Gherkin has to be used by the agent, hence declaring it somewhat self-referential. But it’s a key control of ours that we’re instructing and controlling the agent such that when it does detect a conflict with requirements, that it identifies that conflict, and instead of taking action to modify the requirement as it might otherwise want to do, that instead it’s appending an entry to an open questions file, and then it’s alerting the human that there are open questions that have to be resolved. So this would be a case where it’s trying to implement and satisfy the requirement. It can’t, for some other reason. Maybe the requirements are in conflict, or something’s not specific enough, and it now is adding that open question item that the human can resolve. And once the human updates and answers the open question, then the agent is then free to continue the build. So, as I described before, we have the Givens, the Whens and the Thens, and this is a syntax that is human readable, it’s machine readable, and the agents actually love it and they will behave themselves and follow it, believe it or not.
Megan Graham: I like this rule in particular because it makes me think, if you’ve ever done any kind of design verification, any kind of testing, there’s been that moment where you’re identifying a behavior that is not what’s expected and you’re trying to understand what’s the right answer. Is it the test that’s wrong? Is it the software that’s wrong? Is it the requirement that’s wrong? And so I like the idea of this generating that list of, okay, these are not matching, not calling it as one is correct, but letting the human decide what’s the correct path. That’s kind of like anomaly management in medical device. So I like that particular rule.
Larkin Lowrey: And so let’s talk about where Gherkin fits in this model. And so my model here is a little bit different. I’m not including things like risk management and other things that we would normally have to do. This is just a more compact way for demonstration purposes. But ultimately at the top of the stack, we have user and business needs. Like, what are we trying to accomplish? What’s the goal with this product or feature? What I have here listed under design constraints and decisions might sometimes be considered or called a PRS or a PRD, product requirements. So here we’re documenting what constraints do we have that are immutable, that we can’t change, and then we have also our design decisions that are basically the things that the team is working through to decide together. You know, should there be a mobile app, should there be a web portal, if it’s a web portal, what’s the layout like, all the typical things that you would put in a PRS.
From that we then are producing our software requirements. And each requirement needs to be completely atomic and individually testable, using the EARS methodology as we all know and love. One of the things, and I’m going to demonstrate a little bit of this later, is that we’re using an interviewing technique to actually help us build user needs and the design constraints and decisions and software requirements, where the agent will actually ask us questions to clarify our intent. And we’re able to iterate with the agent to produce those various needs and requirements. And it’s actually very helpful to use the interviewing method for that.
But the point here is that from the software requirements we’re going to produce Gherkin. The AI agents are really good at taking those software requirements and automatically producing Gherkin. We’ll go in and manually tune them and make some amount of adjustment. They’re not perfect, but it gives us a 80% first draft of the Gherkins. We’re also including interface contracts, because that’s also something that we need to control and has to be testable. So thus we’re lumping that together in with the Gherkin. And then the code is just a byproduct. And this is the most awkward and yet powerful point here. And we’ll get to that in a little more detail when we talk about doing spikes. But this is where we see Gherkin fitting in to the overall stack.
Okay. So, some of the controls that we’ve had to put in place, some of them are actually kind of fun to talk about. We had to create write separation. As I mentioned, we don’t want the agents writing requirements, unless we’re actually working with an agent specifically dedicated to writing requirements. We want the coder agent not to be able to do that. And it’s limited to only appending items to the question list. You can’t overwrite them or change them. So we had to put a lot of gates in place to very much control the coder agent’s ability to modify or mutate any document, the document set.
Read separation was equally important. So we had a number of documents — and just for example, the domain model — where the agent would tend to take some of these documents a little too literally, where our intent with the document is to provide high-level conceptual framing, where the robot would — and by the way, I keep saying robot, whenever I say robot you have to think I’m talking about the AI agents — but it would take that conceptual domain model and literally try to materialize it as SQL database tables and whatnot. So we had to constrain what documentation the coder agent can actually see, read and use.
And obviously approval gates. So we do have agents that are able to collaborate with us on developing user needs and the requirements, but we have gates in place such that the actual mutation of those files has to have a human approval for it, right? So we’re not having the agent try to sneak in some changes that we have not fully approved and accepted.
The final one is the most exciting one for me, and that is we spent a lot of time working out all of the methods that the agents can use to jailbreak this entire system. Right? So we have as a principle that any safety controls, any security controls, or even process controls that we have in place cannot be bypassed without going through a specific and defined procedure. And that procedure obviously has to have an appropriate set of approvals and audit trails and whatnot. But the agents, because they’re able to execute arbitrary code, they can create their own code files. They can check stuff in to a repo and run a pull request, modify pull request tests, and run things in your CI/CD pipeline that can actually disable the controls that we’ve put in place.
So we had to spend a lot of time to figure out all of the different ways that the agent could find to jailbreak its way to bypass these controls. And it’s not that the agent is trying to be malicious or devious. It’s just that the agents are trying to be as helpful as possible. And if it has a unit of code that’s not passing the linter, it will often say, well, the linter rule is wrong and I’m going to go change the linter rule so that my code will then be clean. Or it’ll, you know, if a test is failing, it’s like, well, I need to modify the test in order to make it not fail. And so it has certain bad habits like that, and that’s what the tamper resistance is about, is that it’s trying to be helpful. We just need to make sure that it can’t find a way to bypass any of the controls that we’ve put in place.
Megan Graham: This is really important, because this is a question that the FDA asks manufacturers, is how do you know that your AI-enabled medical device isn’t going to be tampered with. And similarly, how do you know that the tools that you’re using in your development process actually work. So this approach helps to build out evidence and having that proof of concept, that you can build out constraints on the agents to help demonstrate and actually have happen that reduced risk of the jailbreaking, of the unintended consequence or unintended behavior.
Larkin Lowrey: Yep. Absolutely. Okay. So, I mentioned spikes before. I’m going to just touch on this. This I think can be an entire webinar of its own. We don’t always know exactly what we want. And this is really the principle of Agile, right? Agile says you don’t know what you want. You need to test in the marketplace first. So these sets of controls that we have for the agents really makes it unfriendly for doing an experimental spike, doing a POC. If you’re doing usability testing, you don’t tweak the UI or some bit of functionality. So we have a provision for having a spike branch where the code enforcement rules are lifted, but all of the document set gates are still enforced. So they can’t make any changes to needs documentation, design requirements, whatever, right?
So the idea here is that you do your experiments in your spike, you test them, you do whatever usability testing, you might go to a focus group, you might do some formative testing on it, you might ask a physician, hey, what do you think. And when you’re done with the spike, rather than trying to figure out how to merge the spike code back into your main code branch, we’re actually using the agents to look at what is different in the spike versus the code branch that it came from. And then it infers, did we add any new needs? Did we discover any new user needs? Did we change how we think about a user need, or are we deprecating a user need? Have we taken away functionality because the user need’s no longer valid? Have we introduced any new requirements? Have we changed a requirement? Right?
So the agent is able to create essentially a documentation change set of, this is what you need to change to your documentation set in order to reproduce this spike result. And so now, instead of pulling over the code from the spike, we’re pulling over the documentation that describes the spike, applying that of course after human review into our main code branch, and we’re throwing the spike away and we’re building now from the documentation the functionality that the spike had described. This is actually a pretty powerful concept. I could probably do an entire webinar on that, but just wanted to mention this because the ability to do these spikes is really important for product development.
Quick point: this should be cheaper, and I’ll tell you why. We’ve observed that it is cheaper to do using Gherkin versus other techniques for constraining the agents. And so the reason is that because Gherkin is structural, it’s constraining in the language and the structure of how you’re creating your test cases and even really as a way to describe requirements. It allows the agent and the LLM to converge on a solution faster. It has fewer bits of ambiguity to resolve. Things are more correlated and coherent, and so it theoretically should reduce and lower token spend on your agentic activities. And we’ve seen that. However, we’ve not done the formal testing for us to be able to say yes it’s absolutely true, and this is how much, and yes it’s true across all agents or only Claude or only GPT, whatever. But this is just a happy accident that this also turns out to be something that would make all of this a bit more economical.
So, I want to show a demo. The demo is a little video about a POC Claude skill where it will interview the user to help define essentially what your product concept is, and its output is Gherkin feature files. And then we’re going to show a one-shot build based only on the Gherkin feature files, what our product concept is, and I’m going to show you what it actually built for me.
So, this is the initial prompt that I’m giving it. This is all of the initial context. And I wanted it to be something silly and fun. So this is a clinical decision support system for patients to use to assess the severity of their funny bone injury. And I’m giving it a lot of instruction here to make sure that it’s continuing to be funny and this is a parody and just meant to be kind of silly. But this, I’m just basically telling it here’s my concept for this product, and it’s supposed to go inside of my patient portal, just for fun. This is all that I’m giving it. So, let me shift over.
Hopefully this video plays, and I will give some voice over as it’s going along. But this is VS Code. I’ve loaded my prompt into Claude Code and it is now chewing through it. It’s loading various files with the skill definition, which describes the interview process and other controls for the methodology that it’s going to be following as part of this skill. But shortly, as it’s thinking about things, it’s going to ask us to review the process that we’re going to follow. So as you can see, here it’s describing this is how it’s going to flow, one question at a time. It’s going to give me options. It’s going to look for boundary conditions, etc. And it’s also giving us a restatement of the vision for this funny bone triage assistant. And it’s asking us, did it capture the intent correctly? And so, hopefully I’ll type yes, and it will move on.
So that would be your opportunity to say, no, no, no, I meant something slightly different, and you can probably write corrections and iterate on it until its statement is reflecting your true intent. So here it’s asking me a question: are there any other persons involved in this other than the patient that bonked their elbow? And so for this purpose, just the patient, nobody else is involved in any of this. And we’re going to move on.
Give it a second. Okay. Next question. Okay. So, what do they walk away with when they finish a session? Right. So do they want just a single joke severity, or also that plus a recommendation? I like the recommendation angle. I don’t think it has to be more complicated than that for this demo.
Now it wants to know more about what to actually do with it. Okay. So, how do we tell the assistant about our elbow bonk? Right. So, I like the multiple choice. It’s a little more easy for a user to follow and demoable versus like picking a number or a slider or something like that. But it’s asking us good questions about how does the user interact with this thing.
And let’s see, what does the made-up severity scale look like? Right? So, is it a number? Is it just a bunch of names? Is it something else? I didn’t want anything number oriented, because I think it’s hard for people to sometimes gauge ranges like this. So, let’s just make it a bunch of funny names.
Let’s see how many tiers. So, in other words, how complex is this measurement scale? And do we have a set of things to pick from? And I said, no, you pick. I just want to see what you come up with as a concept.
Let’s see, which joke tier fits the tone. Okay. So, do we want it related to everyday objects, cartoon sound effects, or deadpan clinical parody? I like deadpan. That’s my favorite. So, I’m going with deadpan. So, it’s giving us a sense of what’s the user experience? What’s the theme feel? What’s the vibe that the patient is going to have as they’re going through this?
Multiple choice questions to ask about to determine the zing grade. So here it’s like, rather than talk like, how can you describe the kind of tingling you feel in your elbow, it was easier to measure what your reaction was, because it’s very much like reflexes. And so I figured the reflex is going to more correctly reflect the severity of your injury. So I just wanted to measure reflexes. So hopefully I will click that in a second in this video.
Randy Horton: Go ahead. It’s okay. Click it.
Larkin Lowrey: There we go. I don’t know why I had paused originally. All right. I’m not going to bore you with watching the whole thing, but maybe we’ll just look at one more question. And then, okay, so here it’s asking, how much interviewing do we have to do? How many questions do we have to ask the user before we actually produce a result? Maybe we’ll look for one more question, because I want to show what happens when you click on Other, because that’s actually useful because you can provide it free form responses that aren’t one of the predefined set of options to pick from.
Yeah. So here it’s the math, right? Are we just averaging numbers, or is it the worst of the three? Is it some kind of a sum and a calculation based upon that? I decided to go with something else, option number four. So I’m clicking Other, and then I’m just going to free form fill in that I wanted to use median value as my calculation.
So yeah, click, type it in. So, I’m providing just my free form opinion of what I want it to be, and we’re off to the races. And so, it will accept that answer and act accordingly.
Okay. So, I’m not going to make you guys continue watching this whole thing. I want to show you actually what it output. So, it took about 25 minutes for me to go through all of the questions. It took maybe 10 minutes to actually build this thing. And so, I want to show you what the build looks like. So, I’m going to show the build prompt just briefly, and then I’m going to go to my VS Code and show you what this looks like.
So, the build prompt is basically telling it, only look at the feature files, plus also look at this out-of-scope document. So, as we were going through the interviewing, I might have declined something. I said no, we don’t need to do that, or this is not something that we need to do. So it’s documenting things that we did discuss, that we did decide were out of scope and were to be excluded, so that the build agent would not try to invent something. But generally speaking, I’m giving it some, let’s maybe call them design constraints here, that we’re not doing any authentication or login of any kind. We’re just trying to show this one screen. We’re not dealing with any kind of a backend. It’s just a single page app, simple client side stuff. Telling it that we want to maintain the tone, the silly tone, etc. And then there’s some additional details. But let me jump now to VS Code and see if I can share my screen correctly, which is always a question.
Randy Horton: So, as Larkin’s pulling that up, basically what we’re doing is we’re doing the things that look like generating requirements, and we’re pulling in user information, and it’s asking us questions and we’re clarifying and that kind of thing. And that should be familiar to anyone who’s done product development, but medical device product development.
Larkin Lowrey: Right. And also keep in mind this is just like a POC, exploration kind of a workflow here. In another webinar I can show you more detail where we’re actually generating our user needs and our design decisions and whatnot. But for this, I’m scoping it simple in this particular case. But as I mentioned, it’s enumerating out-of-scope items, things deferred that we might do later, and then just other concepts that we excluded.
So let’s look at the Gherkin feature file for how we’re scoring this funny bone injury. So we have a statement of like, why are we doing this feature? What’s the point of it? So here we have a scenario. Given the user’s open the tool, the user answers that they uttered an audible “ow,” and answers a creative word salad for what did you say, and then they did a full arm flail, then the result is a grade Z3, severe ouch, right? And because — you didn’t see this, but I told it, well, you can give it randomized recommendations for what to do — and so here it’s giving testing. Well, it’s got to be one of these two things, right? So, it’s giving examples within the Gherkin syntax.
Here are some additional examples of a scenario, right? They pick this kind of yelp, they said this that kind of thing, they did this kind of arm shake, and then it produced a grade of such and such. Right? So it’s providing a set of criteria that the code has to satisfy, each and every one of these tests, and this is all fed into the test automation. So the test automation will run each of these individual rows as a separate test, and we will have a verification report that will show the test result for each one of these things. Given these inputs, did it produce the expected output?
And so, we’ve got a bunch of other feature files that describe other aspects of, like, okay, how do you get there from the portal, whatever that is?
Randy Horton: And so anyway, you get the idea. It’s producing the Gherkin.
Larkin Lowrey: Exactly. Now, and remember, it built this thing only from the Gherkin. It wasn’t looking at any other material, because there really is no other material other than the Gherkin and the out-of-scope. So, let’s see if we can run this thing.
So, here what it started out with was modeling out how you would navigate to this thing from a fake patient portal. I could have given it a screenshot of an existing patient portal so that it would match the look and feel, but it just made something up in order to give us a conceptual hint. So, I’m going to click the funny bone triage assistant and we’ll go through this process. And by the way, one of the questions that I had asked was about, should we be giving disclaimers, and how and where. And so that’s why we’re seeing disclaimer content up above here.
So, let’s start. And I’m going to say I gave an “ow” because it hurt, but it wasn’t too terrible. Normally when my wife does it, she wakes the neighbors. Anyway, so what did I say? I’m not the kind of person that says non-printable things. Plenty of people do. I’m more of like just a mild word, because it wasn’t that bad of a bang. So, what did I do? I do tend to flail my arm around, right? And just try to shake it off in a way. So, I’m going to go with full arm flail. And I want to get my verdict.
So, my verdict is only a Z2. It’s a notable zing, right? So, it’s not terrible, but I guess I should hold off on high fives for a few minutes because that might aggravate the condition. So, I got my application. It’s functional. It doesn’t look terrible. Doesn’t look exciting either, but it’s better than a generic wireframe. And I can go back and do it over again. I can exit out and I get a popup: should I stay and finish it, or should I leave anyway? And so I’m back to my patient portal.
So all of this functionality was built only from the feature files. And if the feature files are generated only from the SRS, then essentially we’ve tricked Claude into building only from the requirements. And that is the punchline.
Let me see, I might have another slide or two if we care. Yeah, I just have one little parting slide. Let me just throw that on screen and then Randy can close this out. So I just had a takeaway, right? And the takeaway is agentic development is the floor, right? We’re not going back. You’re not going to try to shove the toothpaste back in the tube. Not only is that just a fact of life, but it’s actually good. It’s beneficial. It’s actually allowing us to produce better products and really better patient outcomes faster. It’s really helping us to support the Orthogonal mission.
The autonomy that you get with agentic development, which is really what your leverage is in the business of getting better gains, comes with specification quality and enforceability. Right? You have to be specific. It has to be something the agent is going to adhere to and act on. And you have to make sure that the agent is actually going to do the thing and behave itself. And, as we’ve been discussing this whole time, we’re writing specifications in Gherkin, effectively specifications in a language that all three parties read, right? Where the parties are the stakeholder requesting the functionality, the reviewer who is reviewing that it was implemented correctly, and the agent that’s actually going to build the thing. And today, that is the Gherkin language. So, that’s all I’ve got. Back to you, Randy.
Randy Horton: Megan, we’re almost out of time. Megan, maybe you want to give some closing thoughts on this.
Megan Graham: Sure. Sure. Yeah, I’m really excited about this. It really touches on my interest in kind of that formal specification, solving that problem of how do we know that the software works as we intend it to work. You can layer in additional constraints information. For example, if you want to fly under clinical decision support in the US, you can restrict functionality, those kinds of things, or incorporate the four-part test for that. But this really does help to bring kind of the what can be a very complicated process of developing requirements that mean the same thing to everyone. It’s a really nice solution that’s fairly straightforward to learn and understand. So that really is exciting to me, because it really is the cornerstone of understanding what makes really good software that’s safe and effective. So I’m very excited about this.
Randy Horton: Great. Well, thank you very much. If anybody wants to talk about this further, or how Orthogonal may be able to help you on this journey, give us a ring. And if we call you for feedback, please answer the phone. Thank you all for joining us and we’ll see you on next month’s webinar.
After the Webinar
- Agentic AI Playbook: How MedTech Teams Must Evolve
The role-by-role framework behind the workflow shown in this session. - Automating Compliance with AI in the MedTech SDLC
How compliance evidence, testing and traceability become continuous outputs of development. - Agentic AI insights
All Orthogonal articles, webinars and playbooks on agentic AI in regulated MedTech.
Working on an AI-enabled device?
Orthogonal designs, builds and validates medical device software under design controls, from first prototype to post-market change. Explore AI-enabled medical device software development.
CTO, Orthogonal
Larkin Lowrey
Chief Solutions Officer, Orthogonal
Randy Horton
Related Posts
Talk
MedTech Ecosystem Design Controls: Move Faster With the Right Level of Compliance
Talk
Injecting Compliance into Code: Automating Compliance with AI in the MedTech SDLC
Talk
Beyond the Device: Where Digital Ecosystems Are Creating Real Value in MedTech
Talk
Cloud-Native Architecture (What You Should Learn from Amazon, Google and Microsoft for MedTech)
