Faster delivering teams? Kill the test column

A team's board setup can impact their ability to deliver, but by iteratively moving towards an "In Progress" column, teams can foster a culture of continuous improvement and speed up delivery.

👋 Hello! This is a subscriber-only post 🔒 of the Quality Engineering Newsletter. If you want to read the full post, consider subscribing and help support my work.

---

This week's post discusses how particular team choices may lead to unintended consequences. More specifically, how a team organises its work using boards could cause them to be less collaborative and slow their ability to deliver value to their users. It then shows how teams can identify problems with their board setup and iteratively move towards a solution that works for their context. All while simultaneously encouraging the team to work more collaboratively and towards a culture of continuous improvement.

Key insight 

The Theory of Constraints can provide a structured approach to addressing team problems that, with proper leadership support, can serve as a foundation for a continuous improvement culture in engineering teams. 

Top takeaways 

1. Using multiple columns on a board can cause problems such as inner team silos, pushing testing to the end of the development cycle and resulting in teams starting more work than they finish. Boards designed in this way can slow down the engineering feedback cycle and make testing the bottleneck to delivery.

2. Using one "In Progress" column instead of multiple "In Dev" or "In Test" columns can help to reduce these issues. An "In Progress" column can help promote collaboration and encourage team members to focus on the work as a whole rather than just their individual tasks. 

3. Leadership must create an environment that supports teamworking behaviours to help team members iteratively and collaboratively move from multiple columns to one "In Progress" column. 

Topics covered

1. The “In Test” column

• 1.1 What are boards?

• 1.2 How are boards used?

2. Problems with the “In Test” column

• 2.1 Inner team silos

• 2.2 Pushing testing towards the end of the pipeline

• 2.3 More work in progress

• 2.4 Testing becomes a bottleneck to delivering value

3. What do you do?

• 3.1 Add more testers?

• 3.2 Do more automation?

• 3.3 Delete the Dev and Test columns

4. How do you do it?

• 4.1 Just delete the columns?

• 4.2 Remove the columns incrementally using the Theory of Constraints

• 4.3 What is the Theory of Constraints

• 4.4 Five focusing steps of the Theory of Constraints

5. Applying the Theory of Constraints

• 5.1 Step 1: Identify the system's constraint 

• 5.2 Step 2: Decide how to exploit the system’s constraint

• 5.3 Step 3: Subordinate everything else to the above decision

• 5.4 Step 4: Elevate the system constraint 

• 5.5 Step 5: Do not allow inertia to cause a system constraint 

6. Fostering learning environments to support continuous improvement

• 6.1 Structured process for improving team's ways of working

• 6.2 Behaviours that support continuous improvement

• 6.3 Conditions for a Learning Environment

• 6.4 Going slow to go faster

7. Further Reading

---

This post has quite a few images, so best viewed in a browser or Substack app rather than your email client, which is likely to truncate the post. If you prefer a 📺 video or 🎧 audio-only version, then let me know in the comments.

---

1. The “In Test” column

A few years back, a colleague asked me if I could recommend any blog posts or talks that explain why I always suggest using just one "In Progress" column instead of multiple "In Dev" or "In Test" columns for teams. I searched for content online but found nothing that fit the bill. So, I decided to write something myself, which didn't take too long. I also tweeted it out and received a few likes and retweets. However, I soon forgot about it. Later, I found it was the most popular post I had written. This made me feel like I had struck a chord with people. However, I had yet to explain how to move away from the multiple columns and had only highlighted the problems they caused. Therefore, I decided to give a talk to address that and provide more details.

1.1 What are boards?

Image of different types of team boards

Engineering Boards can be digital or physical, but with more people working remotely, the use of physical boards has decreased. They are typically used for teams to show what work is coming up, what they are currently working on and what's been completed. In a nutshell, boards are a way to make teams’ work visible.

1.2 How are boards used?

The way each team uses their boards may vary, but generally, they would start by doing some discovery work to determine what they want to build. The Product and Delivery Managers would then divide the work into tasks and place them into the "To Do" column. When the developers are ready to start working on a task, they pick up the topmost task and move it to the "In Dev" column. Once the developers have finished their work, testers pick up the task and move it to the "In Test" column.

Usually, developers can do more work than testers due to their numbers, so most teams also have a "Waiting for Test" column. This allows developers to keep working while waiting for testers to finish. It also makes it easier for testers to see what they need to work on next.

Eventually, the testers would finish their work and move the tickets to the "Done" column. Then, the product and delivery managers decide which completed tickets should be released to the live environment. The main idea of the board is for work to move from left to right. Not all teams work exactly like this, but it gives you a flavour of how work typically progresses through engineering teams.

Boards like these keep team members busy and make them appear productive. They don't have to search too far for tasks and can proceed with minimal interaction with other team members.

2. Problems with the “In Test” column

2.1 Inner team silos

With boards set up with distinct discipline columns, what you find is that testers spend all the time focused on the "Testing" columns. At the same time, developers stick to the "In Dev" column, and product and delivery managers focus on the "To do" and "Released" columns. When team members start focusing on just their columns, it starts fostering inner team silos, and team members begin to consider work items as theirs and ours. Which in the longer term begins to limit collaboration.

2.2 Pushing testing towards the end of the pipeline

As developers finish work, they push completed items into the "Waiting for Test" column. Then, testers will pick up work from the "Waiting for Test" column as they become available. Now, developers will not wait for feedback from the testers on if there are any issues with what they developed. They will pick up work from the "To do" column and keep themselves busy. The problem with this is that it slows developers' feedback cycle, often finding out days after they have completed the work that there is an issue. Moreover, this working style pushes testing to the end of the development life cycle, making testers the last line of defence before a release and putting them in a gatekeeper position.

2.3 More work in progress

When boards are set up with distinct columns for each discipline, you find that work only happens in two columns, "In Dev "and "In Test ". The "To do", "Waiting for Test", and "Done" columns are all queues waiting for people to do things. Queues like these result in lots of work in progress for teams, even though only two tickets are being worked on. Too much work in progress has many unforeseen consequences. For instance, if testers find an issue, the ticket has to either go back into the "To do" column or straight back to "In Dev".

Now, teams have to decide:

• Do we let the developers switch context and fix the issue so testers can carry on?

• Let developers finish their existing tickets and make the testers wait.

• Or let testers context switch to pick up something new from the "Waiting for Test" column?

Context switching for either side is bad, as there is always the usual spin-up time to get familiar with the ticket. The extra cognitive load context switching entails can lead to people taking shortcuts to get the work done quicker, which usually means even more work in progress. Context switching often results in teams starting more work than they finish, counterintuitively slowing teams' release of value in the long run.

2.4 Testing becomes a bottleneck to delivering value

The only constraint on the developers doing more work is what can be pulled in from the "To do" column and the amount they can do. Given the nature of the work (building software solutions), teams always have more developers than testers, who tend to use the developers to 100% capacity, churning out more and more tickets.

As the developers begin to get through more work, tickets pile up in the "Waiting for Test" column, waiting for testers to become free. Due to teams having a limited number of people to do testing work, testers become the perceived bottleneck in the system, as this is where the tickets are being held up.

3. What do you do?

3.1 Add more testers?

The idea behind adding more testers is to increase your team's testing capacity, which can come in a few different forms. You can increase testing capacity by asking your existing testers to work overtime. Working extra hours can work for a while, but asking testers to do this indefinitely or every time there is a push to release can result in burnout.

You could hire more testers if your organisation has the financial capital. Hiring can work, but there is a high chance that you have a lot of extra testing capacity when things are quiet.

You can outsource the testing again if your organisation has the money, giving you more flexibility in your testing capacity. However, this can slow down the feedback loop for the engineering team and exacerbate the other problems that having an "In Test "column brings.

3.2 Do more automation?

Another way to increase your testing capacity is to use more automation. Automation usually involves trying to automate testing at your product's UI (user interface) level. The thinking here is that if the team can automate more of the manual work that testers do, it will free them up to do other, more valuable work.

Now, automation might help at first and free up your testers. But you've created a new problem: someone now has to build and maintain the automated tests. If that job falls to the testers, you'll still have a testing capacity problem as they are working on something else. Suppose the developers or a dedicated automation engineer picks up the work. In that case, you've still got the underlying issues with inner team silos, pushing testing towards the end of the pipeline and more work in progress.

Simply relying on adding more testers or implementing UI automation testing only addresses the surface-level issue of testing capacity. It does not tackle the underlying issues that may be causing the problem.

3.3 Delete the Dev and Test columns

Another approach which might seem extreme is to delete the "In Dev "and "In Test" columns and move to a single "In Progress" column. A single "In Progress" column immediately removes the discipline silos due to each group having its column. All work happens in the "In Progress" column, no matter what you're working on. Testing is no longer being deferred to the end of the process but is happening as soon as needed. It limits work in progress as you can only push work on if there is someone else to pick it up. Testing is no longer the bottleneck to releasing value. Now, the whole team is the bottleneck due to the limited capacity for people to do the work.

4. How do you do it?