Start with what changed
Testing helps reduce uncertainty, but quality engineering asks what that uncertainty tells us about the changeability of the system.
The other week, one of the engineers I work with asked me what testing they should be doing for a feature they were building.
I didn’t have much context on the feature itself, but my mind immediately went to three things:
Change, uncertainty, and risk.
What has changed? What uncertainty does that change introduce? And how do we minimise and mitigate the risk that comes with it?
I realised afterwards that this is something I often think about, but have probably never written down clearly. So this post is an attempt to do that.
Because “what testing should we do?” is rarely the best first question.
A better question is:
What has changed, and what uncertainty does that change introduce?
Testing is not just about checking the change. It is about providing feedback that helps reduce the uncertainty created by the change.
But the amount of uncertainty a change creates is not fixed. It is influenced by the system we are changing. Some systems are easy to change, test, observe, and reverse. Others make even small changes feel risky. That is where changeability comes in.
What has changed?
I think this is the most important part to understand when thinking about testing. We need to isolate what has changed, why it has changed, and what outcome we expect from that change.
Changes can be broadly categorised into:
intentionally changing an existing behaviour
adding a new behaviour
removing a behaviour
updating the implementation of a behaviour, but with no expected change in what it does
The why of those changes is context dependent, but knowing it helps us understand whether the change matches the expected outcome.
In my example, the developer was adding new behaviour to see if it drove user engagement. The intention was not to affect any other existing behaviours, but to add a new path to an existing core use case.
That's important because the testing we need depends on the type of change being made.
Changing existing behaviour creates one kind of uncertainty. Adding new behaviour creates another. Removing behaviour creates another. Changing implementation while expecting behaviour to stay the same creates another.
The more types of change we introduce at once, the more uncertainty we tend to bring into the system. That is why keeping changes focused can help reduce risk.
Smaller, more isolated changes are usually easier to understand, test, observe, and reverse.
What uncertainty does that change introduce?
Any change to a software system introduces uncertainty because we have gone from a known state of existing behaviours towards an unknown state.
That unknown state is the uncertainty.
The risk is that the system no longer behaves as we expect, or creates outcomes we did not intend.
Some of that uncertainty is about whether the intended benefit will appear. Did the new behaviour improve user engagement? Did the existing journey still work? Did users behave in the way we expected?
And some of that uncertainty is about whether we have accidentally harmed something else. Did we break an existing behaviour? Did we make a core journey worse? Did we introduce a confusing experience? Did we create a problem somewhere else in the system?
So we need a way to reduce that uncertainty to an acceptable level.
That is where feedback comes in.
Feedback reduces uncertainty
To minimise and mitigate risk, we need feedback on the change. Feedback can come before, during, and after the change.
Feedback before and during the change is about minimising risk. Feedback after the change is about mitigating risk.
Before changing something, we need to understand what the change is and what it should and should not do. This is critical because it feeds into all the feedback approaches that come afterwards.
If we misunderstand the change at this point, that misunderstanding can get built in, and the risk can leak through the system.
While building something, feedback usually comes through testing, but that testing can take many different forms.
It might be developers writing code-level tests that tell them whether the code behaves as expected. It might be automated end-to-end tests checking whether the system still hangs together. It might be more exploratory forms of testing, where people investigate the system to see how it behaves.
This is where we try to reduce our uncertainty about the change. Does the system still do what we expect it to do? Does it avoid doing the things we do not want it to do?
You can test forever, so this is where we need to focus on risk.
The two questions I’d come back to are:
Did the intended behaviour change in the way we expected?
Did anything else change that we did not expect?
That second question is important because surprises are often where the risk lives.
After changing the system, some uncertainty will always remain. Some things will slip by us. Things we didn’t consider, didn’t understand, or misunderstood.
So we need ways to mitigate that risk. That might mean phased deployment, telemetry, monitoring, user feedback, feature flags, rollbacks, or carefully fixing forward.
But the point is the same. We need to know whether the system is behaving as it did before, or whether it has changed in the way we expected it to change.
Otherwise, we are relying on hope, delayed user feedback, or uncontrolled learning in production.
Changeability as a quality attribute
The quality of a system is defined by our ability to change it.
– Dave Farley, Fast Flow Conf 2025
This quote gets at the heart of why change is so critical to software quality.
Changeability is not just about whether we can edit the code. It is about how much uncertainty a change introduces, and how quickly we can reduce that uncertainty to an acceptable level.
If we can change a software system with little uncertainty about how it behaves, then the main limit becomes our ability to decide what to change next.
But if every change introduces large amounts of uncertainty and risk, then the bottleneck becomes whatever we have to do to minimise that risk.
This is why testability and architecture are so important. They influence how much uncertainty a change creates in the first place.
Breaking a system down into isolated, testable modules helps contain uncertainty and risk in the area being changed. But if the system is tightly coupled, every change can significantly increase risk because of all the ways it might cascade through the system.
The same is true for observability and reversibility.
If we cannot observe the impact of a change, uncertainty remains hidden. If we cannot isolate, switch off, or reverse a change, then mitigating risk becomes harder.
So when someone asks, “What testing should we do?”, I think the better starting point is:
What has changed, and what uncertainty does that change introduce?
But the deeper quality engineering question is:
What does that uncertainty tell us about the changeability of the system?
Quality engineering is not just about helping teams test this change, but also about helping them improve the system so future changes create less uncertainty in the first place.
Because changeability can be the difference between learning quickly from small, safe changes and getting trapped in a testing matrix hell of our own making.
If this helped you think differently about quality engineering, consider subscribing or sharing it with someone who is trying to make change feel safer, faster and less uncertain for their team.