When quantity leads to quality
Why learning environments shape which approach works best
There is a common belief that quality beats quantity. But a few years ago, I came across a story about a ceramics class that made me question that.
From Art & Fear by David Bayles and Ted Orland:
The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality.
His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: fifty pounds of pots rated an “A”, forty pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot, albeit a perfect one, to get an “A”.
Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work, and learning from their mistakes, the “quality” group had sat theorising about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.
But is this also true for software? If we produce more, does that actually lead to better quality?
I think the quality vs quantity argument is a bit too binary. The environment in which the work happens plays a big role in which approach works better. The clue is in what the ceramics students were actually doing: learning from repeated attempts and from their mistakes.
That difference becomes clearer when you look at the environment in which the work is happening. Some environments are kind, while others are wicked.
Kind and wicked learning environments
The terms come from Robin Hogarth’s paper The Two Settings of Kind and Wicked Learning Environments.
Kind learning environments are more stable. They have fewer variables, a narrower range of possible outcomes, and easy to interpret feedback loops.
An example of a kind learning environment could be chess or golf. Here, the feedback of each move or swing is almost immediate, allowing you to clearly see if you’re achieving the outcomes you’re after, and the rules are fixed for each game and player.
In a wicked learning environment, there are more variables, some of them hidden. The range of possible outcomes is broader, feedback may be delayed or misleading, and it is often harder to see which actions led to which results. Examples of wicked learning environments include stock markets and creative endeavours such as art or writing.
What’s this got to do with software?
The quality ceramics group reminded me of the waterfall approach, which was heavily used from the 90s through to the early 2000s. This is where engineering teams would try to identify all the system requirements up front, do extensive design and planning, and then build the best system for the identified problem.
The problem here is that the feedback loops are too long. The requirements-gathering stage to delivering a working beta could be months apart, at which point what was considered a success may no longer be the case. Or ambiguity in how the requirements were interpreted may not be identified until users see the system.
Software is often built in wicked learning environments where there is a lot of uncertainty in how to achieve an outcome. What the waterfall approach tried to reduce was uncertainty through prediction and upfront design. Whereas agile tried to reduce uncertainty through smaller bets, with fast feedback loops and iterative learning. In that sense, it starts to look a bit like the approach the quantity group in the ceramics class took.
What agile and the quantity ceramics group identified was that trying to think your way towards what your users or teacher considers an “A” grade would always be a guess. Sometimes you’d get lucky, but more often than not you’d guess wrong and have to redo it. So why not try something that’s close to what you think they want and then ask them what they think, or better yet, watch how they behave? Then keep iterating towards the outcomes you and they want.
Essentially, what agile was trying to do was make a wicked learning environment kinder. By shortening feedback cycles, limiting the number of variables in the environment, making the variables more visible, and generally increasing the amount of information available. While this doesn’t guarantee success, it can get you much closer to it and help you abandon ideas that lead you in the wrong direction.
When does the quantity approach not work?
Where this doesn’t work is in situations where you need repeatability and control. For instance, if you need to carry out the same actions perfectly each time, then the quantity approach only matters as long as it helps you perform the action repeatedly. Such as in manufacturing environments.
In these environments, the variables that lead to the creation of the product can be controlled and therefore limited, made more visible, and coupled with tighter feedback loops. All of which makes it easier to understand what needs to happen if the output is straying from the desired result. What mass modern manufacturing did was turn variability into a standardised, efficient, and repeatable process.
In software, especially when solving new problems, the challenge is often less about repeatability and more about reducing uncertainty through discovery and learning. In those situations, it helps to work in smaller iterations so teams can learn faster.
What does this mean for Quality Engineering?
A core concept in quality engineering is the idea of creating healthier environments from What do Quality Engineers do?
By applying a Quality Engineering mindset to make the system healthier, we can nudge and boost aspects to produce the quality attributes we want, which can be the seeds that grow into a culture of quality throughout an organisation.
The reason for nudging and boosting the system is that quality is an emergent property. This means you can’t directly create quality, but you can influence the environment in which that quality emerges. By making the environment healthier, quality engineering can increase the likelihood that quality emerges more often than not.
One of the best ways to create a healthier system is to create a kinder learning environment. That might mean:
Helping teams connect their work to the outcomes the business is trying to achieve, so success is clearer
Working in smaller batches, so there are fewer variables in play and it is easier to see what led to an outcome
Creating clear, consistent feedback loops, so signals of quality are stronger and easier to act on
Kinder learning environments don’t need to be perfect ones. They just need to be good enough that they allow you to learn and minimise the noise that distracts you from that learning. This means you need to be very clear about what you are trying to learn and, therefore, what needs to work well and what can be rough around the edges.
So, which is better, quality or quantity?
In software, that is often the wrong question.
Quantity can lead to quality when it creates faster learning. But most engineering environments are complex. Feedback loops are not always clear, information can be ambiguous, and some of the variables affecting outcomes only become visible later.
So the real question is not whether quality beats quantity, but what kind of environment we are working in and how we can make learning easier.
That is why quality engineering matters. In complex systems, quality emerges from the environment in which teams work. Part of a QE's job is to help make that environment healthier, so it is clearer, faster to learn from, and better able to surface the signals that matter. In other words, kinder.
Further reading: Creating Kinder Learning Environments
If you want to explore this idea further, here are a few posts I’ve written on making systems healthier, improving feedback loops, and helping teams build quality in.
Understanding the system
Nudging and boosting complex systems - A practical introduction to influencing complex systems when direct control is limited.
Quality Engineering - The study of how quality is created, maintained and lost - A way to spot where quality is being constrained in your system and how to respond.
How complex software systems fail - A map of how quality gets lost in complex systems and the patterns behind failure.
Improving feedback and learning
The six layers of testing - Six layers of feedback that help teams see where quality is being lost.
The Testing Unknown Unknowns - A way to think about testing as feedback for reducing uncertainty.
Mapping Feedback Across Uncertainty - Techniques for choosing feedback approaches based on the kind of uncertainty you are dealing with.
Practical ways to build quality in
Building quality into products, processes and people - Six practical ways to help teams build quality into the system.
How to Run a Quality Mapping Session - A workshop format for helping teams define, prioritise, and align on quality.
If you found this useful, you can subscribe to the Quality Engineering Newsletter for more on quality, feedback loops, and healthier systems.


