The Testing Unknown Unknowns

Testing is a feedback tool for reducing uncertainty. By categorising tests into Testing Unknown-Unknowns teams can better understand system behaviours and address gaps in their knowledge.

Testing is often reduced to binary categories—manual versus automated, pass versus fail. However, these limitations miss a more profound purpose: reducing our uncertainty about system behaviour. Shifting from this binary view toward seeing testing as feedback changes how we think about quality and risk within software systems.

So, how can we look at testing as feedback to reduce our uncertainty about how our systems behave?

The testing unknown-unknowns matrix

Taking inspiration from the 2002 United States Secretary of Defense Donald Rumsfeld's statement, "There are unknown unknowns", we can classify testing and other techniques into four main groups:

• Known-knowns,

• Known-unknowns

• Unknown-knowns

• and Unknown-unknowns.

Each level of the matrix requires different feedback techniques, shifting focus from confirming existing knowledge to discovering new insights.