The compatibility of Lean-Agile and medical device software development

In a previous post, we discussed how TDD could be used to boost the quality of medical software. This time, we’re challenging the traditional view that Agile (and specifically Lean-Agile) cannot be used in medical software development. What you’ll see is that Lean-Agile is a perfect partner for medtech product development, and how its practices can increase quality and maintain compliance, all while reducing risk.

Waterfall v Agile

Waterfall naturally leaves people wanting to design and document the entire system right at the start of the project, and then implement the software to match. The downside is that most of the risks remain unknown until near the end of the project. Not least because testing is postponed (and faults not found) until late in development, and stakeholders or users might finally see the product and change their mind about some key features.

In the Waterfall model, defining too many requirements at the start can make it tougher to go to market if they are not all implemented, and the project has less chance of being successful.

Lean-Agile (the flavour of Agile software development we use at Bluefruit, which embraces Lean manufacturing principles) creates medical device software that is of better quality and is responsive to early stakeholder and market feedback. Early feedback happens through early access to the implemented software and regular reviews. Software engineers using Lean-Agile processes can react to new knowledge discovered during development but also document how this changes a project.

Through this early access and responding to discoveries made during development, Lean-Agile reduces risks earlier and enables a product to meet compliance through all stages of software development. (Guidelines are available for using Agile for medical products in TIR45.)

The benefits of Lean-Agile go beyond regular working releases (usually every 2-4 weeks), allowing early testing and ongoing user interaction and feedback. We have identified three key areas where Lean-Agile enhances the software development process for medical devices:

• Quality
• Compliance
• Risks

Quality

As software quality is especially essential for medical device software, where errors could cause harm to people or even death, the following describes some of the ways which make Lean-Agile ideal for developing quality-focused software:

Behaviour-Driven Development (BDD)/Test-Driven Development (TDD)

To ensure that the code that is written is correct and as habitable as possible, BDD and TDD mean writing tests to describe desired behaviour/functionality before implementing code. Doing this means that the tests will pass when adding and refactoring code. These tests can be automated and form part of unit testing or regression testing. Unit tests written as part of TDD mean the tests match the intent of the feature. For more information, see our previous posts (where we expand) on BDD and this one on TDD.

(And if you think TDD takes longer than test-last approaches to development? At first… yes, but test-first practices like TDD significantly reduce the time spent on maintenance and fixing errors in code.)

Pair programming

Two people sharing a keyboard as they collaboratively write code means sharing design ideas and knowledge during the implementation and informally reviewing code during development. Mistakes are more likely to be spotted early, and improvements suggested. Mob programming, where more than two people collectively write code, has similar benefits. Both are a great way to program, that helps to reduce project waste and improve project effectiveness.

Reviews

Not specifically an Agile thing, but an essential part of Agile software practices. A person who wasn’t involved in the development looks over the suggested code changes to check that the quality guidelines have been followed and searches for errors that would cause defects. Ideally, the code and verification tests should also be reviewed against the specific functional requirements to ensure they are testing and implementing the right thing.

Using a tool like GitHub or GitLab means that teams can capture review comments and approvals aiding compliance.

Retrospectives

Regular retrospectives each sprint ensure that teams examine the approach taken during the previous sprint and look for ways to improve working practices and quality. Teams can use outcomes from this meeting to update the software test plan documents and enhance quality in subsequent iterations. They can also be used to address any issues that are affecting overall development velocity.

Continuous integration and regression testing

If automated, software teams run regression, unit tests and verification tests during a build. By doing this during a build, it’s possible to merge new code changes, and defects can be found earlier and won’t be a last-minute surprise. It’s also possible to use code coverage statistics and verification test results for reporting.

Feature prioritisation and preparation for development

Teams and stakeholders plan work for the next sprint, typically lasting two to four weeks, which will see a new iteration of the software developed. Planning the details for such short intervals means avoiding wasted time and effort should features be dropped from the product or plans changed before implementation.

When planning work for an upcoming sprint, a product owner or stakeholder can prioritise the work. Doing this ensures that the work with the most business value or highest risk is done next. These selected tasks can be architected and put into the next planning session.

Planning sessions involve discussing and sizing (estimating effort per task) as a whole development team. The discussions prompt a better understanding of future work. Having this information available before development means the correct thing is more likely to be implemented.

During planning, there’s discussion of planned acceptance criteria and tests, and the documentation is updated.

Any ambiguous requirements are discussed and modified to ensure they are clearer and unambiguous.

Iteration Demos

A demo gives key stakeholders a chance to see the functionality implemented during each sprint, giving early confidence in the software, and the opportunity to provide timely feedback. (It can also help with projects needing to seek further investment.)

A stakeholder will be able to take the version of the software demoed and try it out for themselves at the end of the sprint. Not only is this good for early user feedback, it’s also an excellent opportunity for marketing too.

If the stakeholders want to make changes, this change request can become a new task for the next iteration, with the related documentation, approvals and tests updated along with the change.

Compliance

Medical device software uses the international standard IEC 62304 as a benchmark for medical device software development for compliance in both EU and US markets. From the FDA, TIR45 is guidance which describes how Agile can work for medical software compliance. It describes some of the activities that companies can do at different parts of the Agile cycle to fit in with compliance. Here’s how Lean-Agile assists with compliance:

Traceability

At Bluefruit, we have a dedicated Software Analyst role. They keep track of the requirement test coverage and perform quality audits. They help manage traceability between requirements, code, tests, and documentation to ensure that the requirements are clear enough to be implemented and covered in the testing. Requirement coverage statistics are used to track progress.

As requirements change from user feedback, the corresponding code and tests must be updated to keep them all in-sync.

Alternatively, if the requirements, architecture, and design documents are being written on a per-sprint basis, coupled with what functionality is being implemented, the documents will always match the implemented software. (Our post on Lean-Agile and quality considers how this all enhances traceability.)

Sprint-based testing and documenting test evidence

IEC 62304 doesn’t directly state that each release should be fully tested before deployment, but it does ask that “verification activities are performed and evaluated”—so, really, it does want testing. Using continuous integration that triggers unit and regression tests, each version of the software can have a vast bulk of functionality automatically tested with the results logged.

Integration and system testing can take place at the end of each sprint when adding new functionality, and the additional user testing can also take place at each release.

Repositories

Not specifically a Lean-Agile technique, but Git can keep an evidence trail of the state of the software at each iteration/build. From a compliance and testing perspective, this is important as regulations require tests to be reproducible on the software version that they were run on. Git also logs all changes, including the name of the person who made the change and the specific details of the change. Logging means that all changes can be peer-reviewed (using a tool such as Gerrit or GitHub/GitLab) before merging, with the reviews included as evidence of following quality procedures.

Software development risks

Traditional Waterfall approaches leave lots of risks (unknowns) until near the end of the project when the software is released for testing, and users and stakeholders get their first look. Lean-Agile supports the reduction of risk in various ways:

Fail fast

Agile has a concept that riskiest elements are tackled early (“fast”) so that teams can identify the scale of risk earlier. Early identification enables more accurate planning and mitigations to take place earlier and increases the chance of meeting deadlines.

Verification

Are there any serious defects in the software? Ongoing testing means earlier defect discovery (and fixing), and less test burden or rush to fix significant defects at the end of the project.

Validation

Does the implemented software meet user needs? Don’t wait until the end of the project to find out. Early access to working software at the end of each sprint’s software iteration means key stakeholders and users can trial the software early in development to ensure the software is suitable for user needs and the problems being solved.

Systematic risk mitigation

Nancy Van Schooenderwoert, who has decades of experience with Agile in safety-critical applications, sees Agile as a way to mitigate risk systematically. Nancy holds that Agile techniques such as TDD, continuous integration, unit tests and automation, retrospectives, and so on, are all things which accelerate learning and enable timely knowledge sharing—and automatically becomes systematic risk mitigation.

Improve quality while supporting compliance

Many aspects of Agile and Lean-Agile can improve the quality of software with benefits for stakeholders and users. Risks can be reduced, documentation created and maintained in an ongoing manner, and early visibility of the software means it can evolve to meet market and user needs. Having these qualities, Lean-Agile is an ideal set of principles and practices to use in software development for medical devices.

Written by Jane Orme.