Principle 2 - We can't manage dependencies, we must remove them

Symptom

Even with upfront research, our features are still not delivering the expected value.

Cause

Investing in upfront research reduces our risk of building the wrong thing. However, it does not eliminate the risk. Prototypes are great for getting close to the real experience but they cannot replicate it exactly. How customers behave when they have the product is the real test of whether we have built the right thing. But in many organisations, it can take months to go from idea to production. This increases the amount we need to invest in each test cycle and increases our risk that our product or feature will fail to deliver the expected value.

If we look at the length of time it takes to go from idea to working softare, from the point of view of each piece of work, we can see that the vast majority of the time is spent in backlogs: waiting for a business case, waiting for a decision, waiting for requirements analysis, waiting for design, waiting for dev, waiting for test, waiting for release. We invest a lot of effort in optimising the time it takes to build software but our backlogs are what are really delaying quick delivery.

The Context Change

Before the era of mass manufacturing, products were built by a single craftsperson. The great managerial innovation of the early 20th Century was the separation of the assembly line into distinct specialities where people could much more efficiently complete their step of the build. Breaking a complex process into much smaller skillsets reduced the training time and labour costs which allowed companies to scale much more quickly. The biggest risk with an assembly line is unproductive workers so you need to ensure that there is a backlog of work at each station. That way there is never an excuse for people to stop working.

We replicated this approach in software development. We have a separation of responsibility along the chain from idea to production. There are separate functions who do planning, analysis, design, build, test and deployment among others.

But in mass-manufacturing you are always building the same thing. If you are certain that customers want what you are building the slow lead time for production is offset by the efficiency of operation.

The challenge is that in software development we are always building new things, and we are not certain that customers want what we are building. So the tradeoff between efficiency of oepration and lead time is not the same. Our new goal shifts from the pure efficiency of the build process to reducing the risk that we are building the wrong thing.

Solution

Since the biggest time sink in our process is work waiting in backlogs, reducing the size of our backlogs will have the biggest impact on lead time. With a short lead time in place, we can start delivering much smaller increments of value to customers, get early feedback and iterate towards better solutions.

Impact

Backlogs aren't just there to slow down lead time. There are a few different things that get impacted by a reduction in backlogs:

1. Empty Backlogs: People may run out of work to do because they get through their backlog before the people upstream of them have finished their work.
2. Increased Overheads: The overhead of moving work between specialities increases because, with smaller batch sizes, the number of handovers increases. Things like documentation for handovers, stage gate sign-offs, testing cycles, security sign-offs, change control boards and other dependencies counteract the benefits of the smaller backlog sizes.

Mitigating the Impact

Cross-functional teams

We need to shift from teams which are a collection of specialities to cross-functional teams with no internal backlogs. People work together on items and move them through the delivery process together rahter than in separate funcitonal silos. This appears inefficient because we have highly trained professionals working in areas outside of their core speciality, such as engineers attending research interviews or designers watching developers code, but it is actually more efficient because it enables quicker, and better, decisions as all of the minds are working on the same item together. In addition, it reduces rework because of the increase in quality and it reduces context switching, where people have to remember work they did weeks ago.

Reducing overheads

All of the sign-off steps exist to increase the quality of the product. But there are always mutiple ways to achieve the same goals, each with different tradeoffs.