How software systems fail - Part 2b - Processes

Dr Richard Cook's 18 characteristics of complex systems failure applied to software. Part 2b of 3 focuses on three of the six characteristics, demonstrating how quality is lost at the process level.

Key insight

The introduction of change can lead to new forms of failure, and it is essential to address these potential failures by having the humility to accept that they could be due to the actions of the system, not that of its users. Systems builder's and maintainers' experience with failure is essential to operating complex software systems to lower the chances of catastrophic failure.  

The three main takeaways

1. Change introduces new forms of failure: Being open to the fact that systems builders and maintainers actions may have (unintentionally) introduced them is critical to resolving them when they are high-frequency but low-consequence. Otherwise, you risk them building up and becoming low-frequency but high-consequence.

2. "End-of-the-chain" testing creates issues: Placing testers at the end of the development process slows it down and leads to developers using testers as a safety net instead of focusing on improving their testing processes.

3. Failure-free operations require e…