What the TSB IT meltdown can teach us about software development

Some of the hardest coding environments are in embedded systems and banking systems. TSB and their in-house IT provider, SABIS, had significant issues with an in-house banking platform in 2018. To the tune of 5 million affected customers, weeks of service outages and an increase in fraud attempts on customer accounts.

An independent report from law firm Slaughter and May (link opens PDF), released this week, details the failings of this “Go Live” event and what led to it.

Now, it is easy for us from our cosy corner of Cornwall to look at this report and agree with the issues identified. As the report lays out, a digital transformation “Programme” of this complexity had not been done in the UK before:

“The nature, scale and complexity of TSB’s IT transformation was unprecedented in the UK market.”

There were infrastructure and software changes taking place at the same time. What was being attempted was never going to be easy for anyone to handle. It was always going to be complicated, from the software involved to the compliance and legal demands it would need to meet.

But for a Lean-Agile business like Bluefruit Software, the report highlights the risks we’re trying to avoid when we use a Lean-Agile approach to software engineering and development.

The dangers of relying on a Waterfall model

There are other lessons to be learned from what happened with this project, and you can find them in the report. In this post, we’re focusing on the type of project management methods used.

The report doesn’t go into the types of project methodologies that were used internally. From reading the report, it’s clear that TSB was using Waterfall based methods (like PRINCE2 or PMI PMP).

Waterfall in software is where the creation of a product is developed in a sequence of phases, like the flow of a waterfall, sequentially going downwards to completion. Waterfall focuses on mapping out everything from the start. It doesn’t consider that you might not know everything about the project when you begin and that things can change as knowledge emerges or as requirements change. Deadlines are treated as immutable.

It leaves testing to the end of software development, assuming everything is perfectly planned, designed and built. Testing is just confirming that everything works. The methodology doesn’t account for errors found in testing or as requirements change and how to effectively remedy them—it doesn’t account for extra time and cost needed.

The report found that TSB did account for the idea that “bumps in the road” would occur. So, what did TSB do with the emergent knowledge gained? When they realised that some deadlines were proving difficult to meet, they created “The Replan”, but as the report states:

“The Replan presented an opportunity for TSB to take a realistic view of the status of the Programme and to learn from the Programme’s first 18 months. This opportunity was missed. The Replan was not a comprehensive, ‘bottom-up’ process and there was little attempt to investigate the technical causes of the delays that had been faced to date.” (My emphasis.)

Nothing happened with the new knowledge. It wasn’t communicated or asked for in the ways it should have been, as shown in the report. Instead the project went live with a significant data migration that hadn’t been tested in an environment that genuinely reflected the intended deployment.

How Lean-Agile benefits any software development project

Lean-Agile is a valuable software development methodology for a range of sectors. In embedded software, it’s used in such quality-critical spaces as Medtech and Aerospace.

Lean-Agile in software development shifts the focus of software development from traditional discussions around Scope, Time and Cost, the so-called “Iron Triangle”, with Scope fixed and the rest variables. Instead, it introduces “Quality” to any project, adding a fourth dimension and making it the fixed element with Scope, Time and Cost variables.

Through shifting these priorities and making Quality the most crucial consideration, a software project must take actions that support this. Scope may decrease, leading to software that has fewer features at the start but is stable and reliable and then have features added in future. Time might increase to account for new knowledge. Cost could go down when new knowledge finds more efficient ways of meeting requirements, and bugs found early save money and reputation.

At the core of all of this, the end-user is a constant and can be represented by methodologies such as Behaviour-Driven Development (BDD). BDD considers user interactions, their behaviour and journey through a system to describe the requirements that affect software features and behaviour. Coupled with insights from approaches like User Experience (UX), you can develop user-centric software.

Another key difference here is that testing is something that occurs throughout the development process, and not only towards the end. Developers can use Test-Driven Development (TDD), writing tests before code. Writing tests first means that the developer can get feedback straight away on their code. It enables developers to find issues and address them early. This saves money as errors found in development are usually cheaper to fix than those in deployed applications.

These methodologies and more, which form part of Bluefruit Software’s Lean-Agile approach, can help to avoid and solve many of the issues that can occur in Waterfall.

Okay, maybe Lean-Agile couldn’t have saved the day completely

One of the critical decisions made in the TSB project was to use a pre-existing software package adapted for the UK market. The banking platform (Proteo4UK) wasn’t written from the ground-up specifically for this project.

But taking a Lean-Agile approach would have at least allowed for the project to slow down and reflect safely. Enabling retrospectives, using new knowledge, and encouraging departments to see past silos. Overall, Lean-Agile would have helped scale back the risk involved and decrease the damage that could happen during launch.

Introducing Lean-Agile thinking to an organisation can be an investment with massive payoffs.