Software of unknown provenance in medical devices

What is SOUP in medical device software?

You might know SOUP as “off-the-shelf software” or “third-party software”, and it comes in two key flavours:

• Proprietary software, which another company develops. Windows 10 IoT is an example of proprietary software. Your software team can’t look at the source code behind proprietary software; it is like a “black box”.
• Open-source software (OSS) is software available under a license where the copyright holder allows others to use, investigate, edit, and distribute the software and its source code. Embedded Linux is an example of this.

SOUP means that you know little to nothing of its origins.

SOUP and IEC 62304

It’s that origins element that plays a big role in IEC 62304. In comparison to thoroughly compliant in-house software, off-the-shelf options are unlikely to be fully documented. By fully documented, we mean having things like unit tests or code reviews accessible to your software team.

Missing things like unit tests means that medical device software using SOUP must have measures to mitigate compliance and safety challenges.

And how much you must mitigate the risks involved in using SOUP will depend on the software safety class of your device’s software, according to IEC 62304.

NOTE: Keep in mind, a handful of third-party software options have been developed with IEC 62304 compliance in mind. Some third-party software may not be considered SOUP. Blackberry’s QNX OS for Safety is an example of software where it’s been pre-certified to IEC 62304 Class C.

The three classes of software safety in IEC 62304

Part of complying with IEC 62304 involves picking the correct software safety class for a device’s software.

And this overall safety class includes whichever off-the-shelf software your product uses. So, your product’s rating will depend on whichever of the two (in-house or SOUP) has the biggest risk to patients.

• Class A: No unacceptable risk to health.
• Class B: Injury is possible but not serious.
• Class C: Death or serious injury is possible.

You can find the primary references to SOUP in sections 5.3, 7.1 and 8.1 of IEC 62304. Software in Classes B and C need to meet more of the requirements found in IEC 62304 versus a device rated as Class A.

Tip: It pays to be honest when you’re assigning your safety class. If there is even the slightest chance of an unacceptable health risk: the device software needs to be Class B or C.

But what are the real challenges and benefits of SOUP beyond navigating software classification?

Examples of SOUP in medical devices

SOUP can be used throughout a device and can include things like:

• An Operating System (OS) like Embedded Linux, Windows IoT 10, Android;
• Hardware drivers that control parts of hardware on the device, like USB ports, Wi-Fi;
• Software libraries like TensorFlow (used in Machine Learning) or toolkits like Qt (used to develop modern user interfaces in embedded systems like medical devices).

All the above are complex pieces of software that would take some time to develop in-house or through an outsourced software development team.

Why use SOUP in the first place?

There are many reasons why you might want to consider using SOUP in a device. Bringing in off-the-shelf software can help you with project pace (when you have a robust validation process in place), cost, quality and even safety.

How SOUP can help with software safety through Human Factors and UX

Now you might be thinking, “How can SOUP help with safety?” It might seem counterintuitive to consider SOUP as a way to improve safety. Still, an area where off-the-shelf software can help is Human Factors including User Experience (UX) and User Interfaces (UI). It is incredibly complex to create modern UI from scratch, and so it’s one place where SOUP can help.

Poorly developed user interfaces are a massive problem in medical device software:

User Interface software errors accounted for 46.33% of medical device FDA recalls due to software during 2012-15.

Source

Qt is one such toolkit that you can use to develop a modern UI that puts UX first. But it’s SOUP, so if you were to use it, you’d need to follow the checks outlined in IEC 62304.

IEC 62366 covers medical device software usability and Human Factors, which is part of the standards medical device software needs to meet. (But this post here is more concerned with IEC 62304.)

Challenges of using SOUP in medical device software

The challenges of bringing SOUP into your device might include:

No unit tests

Unit tests are individual tests written for small sections of code. How small? For each component coded into the software, you would have a unit test.

In Agile and Lean-Agile, you typically have software engineers write their unit tests, writing each test before they code. Unit tests form a part of a test-first approach to software development found in practices like Test-Driven Development (TDD).

Unit tests are a vital aspect of the software verification process If you don’t have good unit tests, your software verification is weaker, which means you need to compensate with more validation activities.

(Verification is the process of checking whether you’ve built the thing right (does our product meet the specification) and validation checks whether you built the right thing (does our product have the right specifications).)

IEC 62304 sees unit tests as one aspect of documenting software and showing that features are working as expected.

Off-the-shelf software is on someone else’s schedule

Are there feature enhancements you’re looking to see? Bugs that need fixing or security issues uncovered? When it comes to off-the-shelf, you wait for someone else to deal with these. Your software is partly on someone else’s schedule.

Being on someone else’s schedule is a bigger issue for proprietary software than open source. The transparency and community-collaborative nature of mature open-source software is a huge benefit. It means that once someone finds a security issue or severe bug, it’s dealt with (often) faster than proprietary software. And usually, news of the problem is made public knowledge sooner.

And in the case of proprietary software, discovering bugs and security issues after support ends means you can do nothing about it.

Conversely, if there’s an issue with open-source software and the community is no longer developing that software: you can go right in and fix it yourself.

Features from SOUP may not completely match the features your project needs

Regardless of whether you use proprietary or open source, there’s a chance that what you pick may not fully match up with what you need.

Perhaps the software has lower grade encryption than what you planned for? Maybe a video codec used doesn’t compress footage at the rate needed? Or perhaps it has more features than you need, and your team will have to find ways to handle this?

The features issue is a potential risk, regardless of whether you use open source or proprietary software. You may need a way to navigate this.

The benefits of using SOUP in medical devices

Despite the challenges of SOUP, there are some real benefits to using it:

You’re not “reinventing the wheel”

When you use off-the-shelf software, you are skipping much of the cost, skills and time that comes with software development from scratch. You’ll be able to take advantage of a grown-up software ecosystem.

Yes, testing and documentation will be harder to control. But if you can show you’ve done your due diligence in assessing risks associated with using it, then mitigated against them and done thorough validation: SOUP can be a good choice.

You’ll end up with a validation process you can use again and again

Investing in a robust validation process means that not only will you have that process for the initial product it’s used on: you’ll have something you can use time and time again on other SOUP.

Your validation process will also be something you can use on in-house code as well.

And if you couple this with test automation? You can reduce the level of manual testing needed so that software testers can spend more time looking at usability testing and exploratory testing.

You can make open-source software your own

Okay, this advantage only applies to open source. But yes: if there’s something you need to customise—you can do that.

Imagine you have a legacy product running an older open-source OS. You’re not ready to fully update the open-source software you use. However, you need to bring in a critical security upgrade that is usually only available to the newer OS version. Your team can work to make that security upgrade.

Should the software have poor habitability, your team could take steps to refactor it. Refactoring means making changes to code so on the surface the software experience seems like before refactoring. But the big difference is that the code will have been gutted and completely redone with (ideally) better code.

Your team could even go so far as rewriting that software in a different language if needed. Taking something that’s in Python and rewriting it in C or C++, or even Rust.

Make the most of off-the-shelf software

There will still be use cases where SOUP is not appropriate, and you will need your team to develop that software from scratch. But where risk, cost and time align in a positive way to consider off-the-shelf, here’s how to make the most of it.

1. Don’t be scared

Don’t immediately dismiss SOUP. Make a real effort to evaluate it alongside developing in-house code.

Off-the-shelf software can help you meet the increasing shift to complex medtech. Complexities including the desire for Internet of Medical Things (IoMT) functionality. Plus, the compliance demand for improved usability and user desire for it.

The validation demands of working with SOUP are manageable, and even if you develop in-house, you will still need to have a reliable documentation trail for software verification

And IEC 63204 itself is written with SOUP in mind, so you’re not going against the standard using it. You just need to ensure you assess SOUP properly for risk.

2. Think open-source software first

While there is some proprietary software developed to the rigour of IEC 62304: not all are. Open-source software is a viable alternative that means you can:

• See your source code;
• Make code changes that work with your team, device, and users, rather than against;
• Benefit from many eyes on the source code, making bug and security reporting faster than proprietary or in-house.

Plus, you can easily try it out before working out licences to ensure it’s the right pick for you during the proof-of-concept and prototyping stages.

3. Make software usability a top priority

If there’s only one thing you consider off-the-shelf software to help with: make it usability and ensure it’s part of your evaluation process.

With User Interface software errors accounting for almost half of FDA medical device software recalls between 2012-15, no device makers can ignore usability.

User-centred design practices and processes are also a great way to ensure that what you go with meets end users’ needs and keeps patients safe.