Why TDD is excellent for medical device software development

Before we look specifically at what Test-Driven Development (TDD) is, let’s consider historically typical embedded software projects in medtech.

A Waterfall approach v Agile and Lean-Agile

Traditional software development, Waterfall, leaves software testing to last. While in theory leaving testing to last still assures product quality, the reality is far from this. Bugs and other technical issues are revealed late, pushing deadlines back and leading to expensive fixes.

Based on a prediction-focused development approach, traditional methods of software development have no real flexibility to handle reality. It creates waste and is costly in the long-term. Traditional methods are the opposite to Agile and Lean-Agile software development practices. Agile and Lean-Agile use processes like Test-Driven Development (TDD).

Agile and Lean-Agile focus on adaptation over prediction as a project progresses, using emergent knowledge and continuous improvement right from the start. TDD helps with this.

What is TDD?

TDD is when a failing test is written before any code, we call this a “test-first” approach to software testing. The test will describe the intended behaviour being implemented. When the code is implemented, the test will pass. This helps you to know that the code is doing what it was intended to do. The traditional way is to code first and then test it after fully completing the software (this is called “test-last”), but that is less suitable for medical software development.

Test first v test last

But is it better quality? Does it detect more errors compared to writing tests after the code? When injecting faults using a technique called mutation testing, one study (link goes to PDF) found that software developed with TDD detected a mean of 83.5% of errors, compared to 57.5% when tests were written after the software. The study also found that the variance increased from 10.7 to 26.7, highlighting that there is a problem with consistency when taking a test-last approach.

But what are the benefits of testing first for medical software?

What medtech projects gain from TDD

Here’s what medtech projects gain from Test-Driven Development…

Capture intent

Test-first captures the intent, whereas test-last focuses on the chosen solution. Capturing intent increases the effectiveness of tests. It also increases the maintainability of tests because they are more readable. All this enables tests to add more value because they provide effective documentation of the purpose of the code.

Conversation

Implementing the right thing. Writing a test first encourages the conversation with product owners or stakeholders upfront about what the function should do. How will they know when it is successful? Which requirements will this test cover? Are the requirements clear enough? Test-first creates a short feedback loop.

Instead of stopping mid-implementation to ask questions, or worse—guessing and implementing the wrong thing—the information needed is available at the start, so there are fewer delays, less context-switching and a more precise understanding for the developer.

Another benefit of this is group ownership. Getting the whole team involved at the start means that they share the ownership and are more invested in it succeeding.

Improved quality

This is key for medical software development. Mistakes can risk lives or harm to people. Company reputation is often dependent on the quality of the software it is producing. Because the conversation has taken place at the start, and because a passing test shows the status of the behaviour, you can be more confident that the software is doing what it is meant to.

Including unit testing around the new feature can do all the necessary quality checks—checking boundary ranges, invalid input, and so on.

Reduced waste

Another way that quality is improved is that only code that is needed to pass the test is added. Extra code is removed during refactoring. This makes the code more readable and effective.

Regression testing

Write good tests, and they can be automated. Any time somebody makes changes to the code, the test suite will run and check that all the things that were meant to work still have passing tests. This is an early warning system for changes that might inadvertently break other things.

If you were testing last, you wouldn’t necessarily know about these accidental breakages until right at the end, and have a bigger challenge identifying what caused it. If time is money, this also makes bugs less expensive when found early.

Traceability

Test cases can be linked to the requirements or user stories that they are implementing, which you can provide as evidence for regulatory compliance. Stats can be produced for each requirement to show the pass/fail status of each.

Risk awareness

Early exposure of failures can give a more accurate indicator of product health. If there are many failures in a particular area, this can form the basis for a risk assessment for regulatory compliance.

Use unit tests and TDD to bolster quality and traceability

Well written unit tests, as part of TDD, can be a helpful and lightweight way to support the verification of your software design. Test titles from unit tests should clearly explain what function that unit of code is performing, which can be traced back to the requirements of your software.

Simple and effective ways to maintain traceability throughout your source code lessen the reviewing burden on the development team. Testing, including TDD, and reviewing code and design specifications are both safety nets to the overall code quality. Tracing unit test titles back to user stories helps to identify interrelationships within the code base and can provide useful evidence of compliance with IEC 62304.

Taking account of the complexity of software development for medical devices, and the need to safeguard code quality to help ensure devices are safe for patient use, including Test-Driven Development in your software development plan is a positive step towards software with integrity and demonstrable quality.

Written by Jane Orme.