I’m a developer who is fascinated by the delivery flow in project management systems. I’ve spent a lot of time thinking about it, and read books like Agile Management for Software Engineering, The Art of Agile Development, Slack, and my personal favorite, The Phoenix Project.
The Phoenix Project has really opened my eyes to the many inefficiencies we face. I’m also really fortunate to have had the opportunity not to just study different delivery methods, but to practice them by managing delivery teams.
For this article I’ve decided to combine my two passions: software engineering and delivery flow.
In fact, I’ve ended up building a basic throughput simulator!
What You’re Going to Learn
- Eight major inefficiencies that increase wait time
- How the following delivery flows perform in practice:
- Waterfall
- Scrum
- Kanban
- Why everyone should chill out and give each other some slack
- Why wait time is the root of all evil when it comes to efficiency
- That it’s all about the team, not the individual, when it comes to delivering software
The 8 Wastes in Software Development
Before we start, it’s important that you understand different types of waste in software development.
This list is not comprehensive, and I am sure that someone could write a book on this that runs hundreds of pages long. But this list will give you the main models you need to consider.
1. Transport
These inefficiencies occur when you are moving artefacts around. Perhaps you’re submitting code to several branches, or cherry-picking check-ins.
2. Inventory
This is when produced artefacts are not being used.
Artefacts such as:
- Requirements
- Architecture designs
- Graphics and wireframes
- Proof-of-concepts
- Code
… are produced weeks or months upfront. The moment an artefact is produced, it’s out of date and needs changing.
3. Motion
These inefficiencies occur during the creation of artefacts.
Most developers need to juggle many tasks at the same time. There’s a cost that comes with context switching and multitasking.
Maybe you need to keep track of different software versions, projects and roadmaps. Or people are constantly calling you, or coming over to see you about something.
You need to pick up work that you haven’t touched in days, months or years, and you need to relearn what you’ve already done.
4. Waiting
You’re waiting for your check-in to build. Or you are waiting for artefacts to be produced.
Your IT infrastructure is flaky, your computer stops working, the internet goes down, or the email server is not working.
Your testers in the team get back-ended. They’re waiting around for work for days, and get it right at the end of the iteration.
You have discovered that you have misunderstood the requirement — now there is more work to do.
Or maybe you’re waiting on someone to review your code.
5. Overproduction
You’ve produced your software, but you’re not shipping it. It’s sitting on the shelf and nobody is buying it. The work you’ve produced is no longer needed.
6. Over-processing
This happens when you produce artefacts that weren’t requested.
You’re polishing your code and adding in extra extension points for those amazing features that’ll be built in the future.
You actually working on something that nobody asked for, and you are doing it because it’s “cool”.
7. Defects
This occurs when your artefacts have errors in them.
What you have produced doesn’t meet the agreed acceptance criteria. Or maybe what it doesn’t meet the technical standards (performance, security, maintainability).
8. Skills
This occurs when the producer of the artefacts is being under-utilized.
Perhaps you have not been given adequate training to do the actual job. Or you can do much more complicated work… but it’s not being given to you.
What Is Slack?
Yes, it’s a messaging app — but in this case, I’m talking about an influential book called “Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency.” It’s 209 pages long — I’m not going to do it justice, but here’s a summary:
In order for your teams to be happier, less burnt out, more innovative, and for your organisation to respond better to change, you should add “free time” to your delivery process.
When you plan your iteration, you just add 10%, 20% or whatever percentage of time you decide for innovation.
Our simulation is going to put this theory to the test, and see if it improves delivery rate.
The Simulation
I’ve created a set of simulations for three project management methodologies. You’ll see them below, as well as a chart that will compare the throughput speed of each delivery approach.
Rules
This simulation was simplified for my benefit and yours. The ball in the queue can either move forward or just stay where it is (skip a turn). The ball in the queue can’t move forward if there is a ball in front of it. If the queue is full, a new ball will not be added to it.
How Are These Rules Relevant?
You might be thinking: why would the ball just stay in one place, why would it skip a turn? Well, I’ve tried to simulate the real world.
Remember the eight wastes? They all cause wait time, and this is why the ball might choose to skip a turn.
But this means that all the balls behind it can’t move forward. They are blocked. When ball is blocked, it will flash.
In the real world, things are also messy and a bit random. I’ve added some random blocks to all delivery flows to make things more realistic.
For example, Kanban should, in theory, perform amazingly as it minimizes many of the 8 wastes — but we are human, we make mistakes, and customer priorities change.
Let’s simulate some delivery approaches!
Waterfall
This is a sequential batch approach. One team of a particular discipline does some work and passes it to the next discipline, and so on until you ship.
Typically there is very little feedback, and when there is feedback it’s very late in the process.
People are mass-batching work and passing it down the chain. However, very often, this work comes back up to the original author before going down the chain again.
Regular simulation
See the Pen TSWaterfall by SitePoint (@SitePoint) on CodePen.
Simulation with slack
See the Pen TSWaterfallSlack by SitePoint (@SitePoint) on CodePen.
Kanban
This is a single piece flow approach. Cross-discipline or component teams are working on something, and release it immediately.
Feedback is immediate, and work is added and removed in an ad-hoc fashion.
Regular simulation
See the Pen TSSinglePieceFlow by SitePoint (@SitePoint) on CodePen.
Simulation with slack
See the Pen TSSinglePieceFlowSlack by SitePoint (@SitePoint) on CodePen.
Scrum
Scrum is a parallel batch process. Cross-discipline teams are working on a problem at the same time for a short while, and then they ship their software.
Teams have retrospective meetings, planning meetings, grooming, and so on. Time is allocated for preparation and improvement.
Regular simulation
See the Pen TSBatch by SitePoint (@SitePoint) on CodePen.
Delivery Rate Comparison
This chart shows the number of balls delivered across every delivery approach after 10,000 iterations have passed.
What Can We Learn From This Simulation?
1. We need to chill out and cut each other some slack.
In any system where there is randomness, we should factor in some slack.
The more randomness in the system, the more slack you need to add.
By adding time you reduce randomness, which reduces the interlocks, which reduces the lead time. The slack time just allows you to deal with things better. However, you are paying for it — with time.
But the idea is that paying for it with time actually gives you a lot more time back. People make fewer mistakes, have higher quality discussions, are happier at work, share knowledge better, do less rework, and they don’t quit as often.
But it’s still a lot better to have a good, strong process in place, where randomness is eliminated as much as possible and there is a bit of slack factored in as well.
2. We need to reduce the wait time to increase the delivery rate.
(Delivery Rate) - (Wait Time In The System) = Actual Delivery Rate
This equation sounds painfully obvious.
But look around you. In your organisation, are things are setup to reduce wait time? The answer is probably going to be: no.
How do we reduce wait times? Here are a few tips:
- Have fewer projects going on at the same time.
- Make sure that delivery teams are small. Big teams require more communication and more management, which is all pure waste.
- Ensure that delivery scope is super clear by using INVEST criteria.
- Deliver just-in-time.
- Deliver just-enough.
- Give people time to do the actual work. Reduce the number of meetings and general interruptions.
- Break work down into very small chunks, so that people feel and see progress.
- Break work down into very small chunks, so it’s easier to estimate and plan.
- Give people more time to do something than they’re asking for. The worst case is that they will take longer because something came up (they discovered more work, got interrupted, etc). The best case is that they will actually deliver early and feel motivated to get onto the next item.
This list is not intuitive. It feels like we should do the opposite, as we live in a society where people think that by putting a huge amount of pressure on people, cutting down their delivery estimate, they can deliver faster and better quality work.
I blame self-help books for miscommunicating Parkinson’s Law: “work expands so as to fill the time available for its completion.”
This sentence is not a scientific ”law”, was not proven with evidence, and was taken out of context. People also often forget that being effective is a lot more important than just being efficient. People need time to think, in a positive environment, so that they can be effective.
3. It’s all about the team, and not all about the individual.
An important concept to remember: when you are in the system, it’s not about your individual delivery rate. It’s about the delivery rate of the overall system. Let me explain.
You might be a super efficient individual. You might work around wait time as you do overtime because you really love your work.
Let’s say you are programmer and you are doing this.
If you are not getting requirements from business analysts fast enough, you might end up guessing what you need to be producing next. As you are writing code almost blind, you are creating inventory, defects, and you are almost certainly overproducing.
You might want to refactor some code because it’s not that good. It has been sitting on the shelf for a while and you have not shipped it yet, but you just don’t like how it looks. In this case, you are most likely over-processing and creating waste.
Imagine you are a back-end developer and front-end is not yet ready.
Guess what will happen? You will start working on some other feature while the front-end developer completes his work.
This means that you are now working on two features. What if the front-end guy gets sick? Now you are working on three, four, five features.
In the meantime, no real value was added to the organization. Why? Because nothing was shipped, and nobody is using what you have written.
You are just building up inventory, and you are not learning anything as you are not getting any feedback from your customers.
Believe it or not, by being a super efficient employee and working on the wrong thing, you can actually do more damage then good.
If you really want to work hard, that’s fine! Refactor shipped but buggy code, improve quality of service (performance, security, etc), pair up with a junior engineer, improve internal processes, or do some research instead.
When you are in the system, you are only as strong as the weakest link in the system.
So fix the weakest link the system and enjoy a better delivery rate.
This is where the Theory of Constraints (ToC) comes in read my ToC article here.
Conclusion
The more randomness there is in the system, the more slack you need to factor in to improve the delivery rate.
Find the bottleneck and do whatever you can to reduce wait times to improve the overall delivery rate (this is what the Theory of Constraints teaches us).
Remember the eight wastes and ruthlessly remove them from your processes.
If you’re interested in seeing how these simulations work or would like to adapt them to create your own simulations, you can find the code on GitHub.
Frequently Asked Questions (FAQs) about Waterfall, Kanban, and Scrum
What are the key differences between Waterfall, Kanban, and Scrum?
Waterfall is a linear project management approach where each phase of a project must be completed before the next one begins. It’s often used in projects where outcomes are well-defined and changes are minimal. Kanban, on the other hand, is a visual system for managing work as it moves through a process. It allows for continuous delivery without overburdening the development team. Scrum is an agile process most commonly used for product development, especially software. It’s a flexible, holistic product development strategy where a development team works as a unit to reach a common goal.
How does the flexibility of Scrum compare to Waterfall and Kanban?
Scrum is more flexible than Waterfall as it allows for changes and adjustments throughout the project. It’s based on iterative and incremental progress, where requirements and solutions evolve through collaboration. Kanban also offers flexibility, but it’s more focused on continuous improvement, flow, and process rather than on an end product or project.
Which methodology is best for my project: Waterfall, Kanban, or Scrum?
The best methodology depends on the nature of your project. If your project has clear, unchanging requirements, Waterfall might be the best choice. If you’re looking for continuous delivery and want to improve your process over time, Kanban could be the right fit. If your project requires flexibility, adaptability, and high customer involvement, Scrum would be a good choice.
Can I combine different methodologies like Scrum and Kanban?
Yes, Scrum and Kanban can be combined to create a hybrid methodology known as Scrumban. This approach takes the flexibility and customer involvement of Scrum and combines it with the visual workflow and continuous improvement of Kanban.
How does communication differ in Waterfall, Kanban, and Scrum?
In Waterfall, communication is more formal and structured, with set communication points throughout the project. In Kanban, communication is continuous as team members update the Kanban board and discuss improvements. Scrum encourages daily communication among team members through daily stand-up meetings.
How does the role of the project manager differ in Waterfall, Kanban, and Scrum?
In Waterfall, the project manager has a central role, overseeing the entire project. In Kanban, there’s no defined project manager role; the team is expected to self-organize and manage their own work. In Scrum, the Scrum Master acts as a facilitator, helping the team follow Scrum practices and principles.
How do Waterfall, Kanban, and Scrum handle changes?
Waterfall is less accommodating to changes once the project has started, as it follows a linear path. Kanban and Scrum, being Agile methodologies, are more adaptive and can accommodate changes more easily.
How do Waterfall, Kanban, and Scrum approach testing?
In Waterfall, testing is a distinct phase that occurs after the development phase. In Kanban, testing is integrated into the workflow and happens continuously. In Scrum, testing is done within each sprint, allowing for regular feedback and adjustments.
How do Waterfall, Kanban, and Scrum deal with risk and uncertainty?
Waterfall tries to mitigate risk through careful planning and design. However, because it’s less flexible, it can struggle with uncertainty. Kanban and Scrum, with their iterative approaches, are better equipped to handle risk and uncertainty as they allow for regular feedback and adjustments.
How do Waterfall, Kanban, and Scrum impact the speed of delivery?
Waterfall can be slower as it follows a linear path and each phase must be completed before the next begins. Kanban focuses on continuous delivery, which can speed up the process. Scrum works in sprints, which can also accelerate delivery as work is broken down into manageable chunks.
Zan KavtaskinZan Kavtaskin is a software engineer and architect. He works at MHR as a Software Design Director and he has also written software for companies such as Experian, Emirates and RoyalMail.
