Four reasons why your Lean-Agile projects run over budget and over schedule

Statistics can be helpful. You may have heard that Agile delivers a six-times higher success rate than Waterfall (CHAOS report 2015). And so, this only adds insult to injury if you’ve invested in Lean-Agile or just Agile, but your projects are still running over budget and over schedule.

So, what’s going wrong?

Here are four questions that might help diagnose your situation:

1. Must we?
2. Can we?
3. Do we?
4. Should we?

1. Must we?

Must we fix the budget and schedule? Are we creating unnecessary project constraints that are setting the project up to fail?

There are two parts to the equation when assessing if a project is within budget: the size of the budget and how much it ends up costing. The business usually sets the budget. If the budget is too small, then the project is set to fail before it starts. You can say the same for deadlines.

A standard approach is that the business defines a scope. Then the engineering team provide an estimate, and then (if acceptable) the business converts that estimate into a fixed budget/schedule.

Every one of these steps has the potential to harm an Agile or Lean-Agile project.

Defining a scope at the start of the project can create barriers to responding to emergent knowledge. You can manage this by frequently reminding all stakeholders that the scope will change.

Estimating is inherently inaccurate at the start of a project but gets more accurate as you progress. If you must lock down an estimate at the beginning of the project, then this will increase the probability of failure.

Fixing the budget at the start of the project introduces significant business risk. It ignores the reality that:

• Key facts might be discovered during a project.
• Or that high-value feature opportunities might emerge which would lead to ten times more sales.
• Or that a technical problem might emerge which makes the project unviable.

Don’t ignore these facts.

The answer is simple: communication—high-bandwidth communication between the engineering team and stakeholders:

• Have a feasibility stage.
• Create review points throughout the project.
• Re-evaluate the business case and cost/benefit at these points.
• And allow for a large error margin on early estimates.

(Check out our post on Time-and-Materials Contracts for Lean-Agile run projects for more information on how to balance scope, budgets and estimates, and how to get communication working between stakeholders and software teams.)

2. Can we?

Can we adapt the project plan to respond to emergent knowledge?

Agile project management practices have been designed to make this possible, but you must view them holistically to understand if they are barriers or catalysts to adapting. Look at the three project life cycles below (borrowed from AAMI TIR45). You can describe all three of these life cycles as Agile, but:

(Click image for larger view.)

This first staged delivery lifecycle does not allow you to deploy a release when you reach your deadline or your money runs out. Coherent, user-focused releases don’t seem to be part of the project plan.

(We would strongly recommend using story mapping as a technique to coordinate better your Agile or Lean-Agile project management practices (such as sprints and user stories).)

(Click image for larger view.)

The second diagram is much better. This approach enables the product owner (or project manager) to trigger a release at a time that fits business requirements. But we are not getting all the benefit of Agile or Lean-Agile.

(Click image for larger view.)

The third diagram here is the best approach of the three. Not only can we trigger a release to meet business demands, but we are also maximising the benefit of having regular, coherent, user-focused releases. We show these releases to users and stakeholders, enabling high-bandwidth conversations about the costs and benefits of features and technical decisions.

Note that the success of this evolutionary project life cycle also depends on good Agile technical practices. When you’re adapting your requirements and your project plan in response to feedback from users and stakeholders, you introduce the risk of poor-quality software. The best mitigation to this risk is to successfully use practices like Test-Driven Development, Behaviour-Driven Development, pair programming and living documentation.

3. Do we?

Now we can adapt the project plan, but do we exercise that power?

The keyword here is ruthlessness. Are stakeholders prepared to cut low-value features? How are these decisions made? It is easy to add features to a backlog, but it’s hard to remove them.

Where you have an individual responsible for the backlog who has the true authority to remove features, it is easier, but this is not always possible. More commonly, the product owner will need to work hard to understand which features are low value. And then, work even harder to get the stakeholders to agree which features get axed.

If you can get all the stakeholders together in a well-facilitated user story mapping workshop, it is much easier to make well-informed, ruthless decisions. Using story mapping means all the stakeholders can see the level of ruthlessness required to achieve budgets and deadlines.

4. Should we?

Now we have the ruthlessness to deliver the project on time/in-budget, but should we?

The evolutionary project life cycle and tools like story mapping enable us to deliver on time and on budget. But what if our drive to deliver within our budget has led to cutting high-value features?

A vital benefit of this Lean-Agile/Agile approach means we have a choice. We can choose to cut the feature. We can also choose to add one more sprint. Or we can choose to save the feature for the next version.

How do we make this decision? Again, the key is good communication. Bring marketing together with finance, together with engineering. Discuss the impact on the budget and deadline. Discuss the benefit it will bring to the business. Provided you present all the facts to the group, a good decision should emerge.

What to do next when your project isn’t going as expected

The next time you are reflecting on why you are overrunning budgets or missing deadlines, ask:

1. Must we fix these budgets and deadlines? Can we be more fluid?
2. Can we adapt? Do our processes allow us to?
3. Do we adapt?
4. Should we stick to the budget/deadline?

If you can apply an Agile/Lean-Agile mindset to all these questions, your projects will be more successful.