Quality: the Lean-Agile way to accessible, safe software

There are strategic business benefits to pursuing quality code. Quality in Lean-Agile means software that works well for both the users using it and the engineering team building it, which brings down waste in the long-term.

In this post, you’ll find out how quality and safety go together. We’ll explain what quality software looks like and how you can use Lean-Agile to increase quality in your software projects.  You’ll learn about:

• Perceived integrity
• Conceptual integrity
• The importance of habitable software
• Starting to think about quality in your organisation

In safety-critical software, quality is essential, but often overlooked

Do you remember the problem Toyota had with “unintended acceleration” in some of its vehicles—a fault linked to many deaths. It took an investigation by NASA and then one by Barr Group to find out Toyota’s software was putting people’s lives at risk.

How about hackable insulin pumps? Medtronic’s MiniMed 508 had to be recalled in 2019 after Medtronic found the pumps had security holes which could not be patched.

And the continuing saga with the Boeing 737-MAX? Software is seen as the leading cause behind several crashes that have left hundreds dead and planes grounded.

In each of these sectors—automotive, medical devices and aerospace—guidance and standards exist which software ought to meet.  As things stand, there are loopholes—MISRA-C is voluntary guidance for car manufacturers, Boeing were trusted by the FAA to provide safety assurances on their products and Medtronic didn’t consider cybersecurity early enough in the development process to mitigate the risks.

But the Barr Group’s investigation of Toyota’s software finds that compliance only shows one side of quality. Toyota’s affected software was so low quality it was dubbed “spaghetti code” in court, with many critical features proven to be “untestable”.

The not-so Lean-Agile approach to quality a.k.a. quality is just a tick box exercise a.k.a. big design upfront

Sam looked at the email in front of them. The words on the screen said the software their team had been developing for a medical device was “utterly unusable”.

A year of hard work was slowly circling towards a metaphorical drain. Sam had done their best to make sure the team met the user requirements that had been planned from the beginning, while also meeting the demands of working to IEC 62304. Surely meeting all this was enough.

With user trials, they were supposed to find that everything was at least working as planned. Finding they had usable software, perhaps only needing to make a few tiny tweaks to the user interface.

The team had done their utmost to develop features which mitigated risks. But while the software was compliant, the user trials found it was useless to the people it was intended for.

For a moment, Sam had considered asking for the budget necessary to add features the users were now saying they needed. But then Sam started adding up the costs involved and felt a little sick.

What Lean-Agile means by quality in software

For Lean-Agile software engineers, quality goes deeper than just functioning software that fits compliance and user requirements.

Mary and Tom Poppendieck in Lean Software Development: An Agile Toolkit frame quality as perceived integrity and conceptual integrity:

• Perceived integrity involves the elements of the software or product the end-user sees and interacts with. It includes the display and user interface they engage with, and how it feels to use it.
• Conceptual integrity includes the elements that go on beneath, things the user will never see or directly engage with.¹

Lean-Agile principles support maximising quality through “building integrity in”² and development teams need business leaders who understand and support this.

Perceived and conceptual integrity: from the roots to the leaves

It helps to think of perceived and conceptual integrity in software like a plant. Above the soil, we have the parts of the plant you can see—the leaves, stems, flowers, fruit. Below the earth we have the plant’s roots, that you can’t see but are critical to the plant’s survival and growth.

Leaves: Perceived integrity

These elements determine if software has perceived integrity:

• Functional software works and performs as expected, with each feature working as specified.
• Reliable software shouldn’t crash often or have errors that stop it from working.
• Economical software is efficient, providing the most straightforward solutions to problems so that users can do tasks with minimal effort.
• Usable software is focused on the user and can be easily understood and used by them.
• Meaningful software only has features that provide real value to the user, features that don’t offer value shouldn’t be included.
• Pleasurable software is enjoyable to use; the user feels a sense of satisfaction and pleasure by using it.

You can think of these as a hierarchy of user experience needs like this:

The minimum need is for the user to find the software is functional. Without this, it’s impossible to meet the rest.

Roots: Conceptual integrity

Conceptual integrity comes from these three core elements:

• Habitable software has source code that’s a pleasure to work with. Its clear structure means any developer can make changes with confidence, and it’s generally easy to go into. The accompanying documentation is also clear and self-maintaining.
• Maintainable software has source code that’s written clearly and has a robust test framework, making it easy to keep the software working with future security and bug fixes.
• Scalable software is structurally sound, meaning it allows for new features and product evolution in the future.

The essential element of conceptual integrity is that software is habitable. This is the part that Lean-Agile software engineers are most concerned with because it affects everything.

“But I don’t need to ‘build integrity in’ to comply with regulatory standards.”

You’re right. You don’t need to ensure quality by building integrity in to comply with software regulations. There is no obligation to bring Lean-Agile thinking to it at all.

But when you look at standards (like IEC 62394 for medical devices), you’ll see these standards often hurt the habitability of software and subsequently its conceptual integrity. Although it ensures traceability to user requirements, there is little focus on ensuring you have the right ones, which hurts perceived integrity.

It does little to encourage the kinds of feedback and actions that could stop these issues. Also, if coupled with traditional Waterfall methods, it amplifies this damaging effect.

And while risk mitigation needs to happen, listing out all mitigations for risks at the start and using those to create features doesn’t add value. Risks need to be prioritised by their potential to cause harm; a lack of prioritisation creates waste without evaluating them.

How habitable software helps quality

Habitable software requires more work as you go, but it pays off in the long-term.

There are specific activities that software engineers can be doing as they go along that will make software more habitable. But they need the time and resources to complete those habitability activities.

Habitable code through refactoring

Habitable software happens through activities like refactoring. This is where software engineers optimise a feature that’s already been developed and meets acceptance criteria. Optimisation leads to:

• Code that’s easier to work with on a day-to-day basis, including by new team members, because the code is easy to read and understand.
• Clearer documentation that people from inside and outside the team can read.
• Software architecture that’s more maintainable and scalable, making it easier to test and add features in the future.

Future-proofing your software and your documentation

There are many reasons your team may need to revisit a product’s software and documentation. From changes in hardware to fixes for bugs and security holes, to adding new functionality to keep the product relevant. Or due to new regulations and compliance needs.

Currently, companies selling medical devices in the EU are working hard to meet the European Union Medical Device Regulation (EU MDR) 2017/745. Recertifying existing products (and certifying currently self-certified ones) is a great deal of work.

Following a Lean-Agile approach to software development would have helped make handling EU MDR easier for product teams. It can help through:

• Traceability between requirements, code, tests and documentation
• Sprint-based testing and documenting test evidence throughout the development process

Retain a motivated team

Maximising conceptual integrity means development teams work on code that is a pleasure to work with. But also, the kind of environment that enables teams like this has the added benefit of being a workplace people want to work at. This helps them perform better.

Bluefruit Software finds that by having a happy workforce, your projects will be more successful, and you’ll have more satisfied customers and users as a result. We certainly see it as a key part to being named Cornwall’s Growth Business of the Year in 2019, as well as Cornwall’s Best Place to Work, and the South West’s Best Technology Workplace—all in 2019.

Support the roots, and the plant will flourish

When you’re making a case for empowering teams to deliver habitable code, enabling conceptual integrity, it’s good to make the roots easier for stakeholders to see.

Talk with development teams about ways to visualise those roots, the conceptual integrity, to help decide what kind of metrics could be used to measure the roots’ health.

Some things to ask the development team include:

• How many lines of code are there in a file?
• How many statements are in each function?
• What is the complexity score of the functions?
• What percentage of the code contains comments?
• Have they used labels instead of “magic numbers” in the code?
• What percentage of the code is covered by unit tests?

Ask the development team what they would like to be doing to improve the health of those roots.

There are many ways these metrics can be shown and explored, including tag clouds, heatmaps and #include maps.

A note on function complexity

A 2014 case study by Professor Phil Koopman shows (link opens PDF) just how important conceptual integrity is. Discussing a key cause of the Electronic Throttle Control System code which led to Toyota’s “unintended acceleration” problem, the professor showed it had a function complexity score of 146. The code was 1300 lines long, with no unit tests and contained over 67 functions with untestable complexity. A score over 50 is seen as “untestable”.

Bring the idea of quality to your organisation

Once you have your metrics, you need to work on gaining support across your organisation:

• Explain to your organisation what you’re trying to do and why (coach Lean-Agile principles).
• Make quality a strategy, so that all parts of the business are working towards a common goal.
• Allow stakeholders access to metrics which have been agreed with development teams.

When everyone is pulling together, you get a multiplying effect that positively impacts projects. For this to happen, you need to communicate why quality is essential.

Further reading

Check out the posts below to learn more about how Lean-Agile can benefit your teams and product development.

References

¹ Mary Poppendieck, Tom Poppendieck, Lean Software Development: An Agile Toolkit (Crawfordsville, Indiana: Addison Wesley, 2003), p.125.

² Mary Poppendieck, Tom Poppendieck, Lean Software Development: An Agile Toolkit (Crawfordsville, Indiana: Addison Wesley, 2003), p. xxvii.