What quality attributes do developers care about?
Exploring Google's Theory of Software Quality: Lessons from a Software Engineer's Perspective
Key insight
Google's research on a theory of quality matches up with my views on building quality into processes and products. Specifically, process quality aligns with my view that building quality into how teams work can improve product quality. However, the theory differs in that they break down product quality into three distinct areas: code, system and product, bringing a more nuanced view of product quality.
Top three takeaways
1. Researchers found that Developers think about several quality attributes when they think about quality. These are defect rates, reliability, maintainability, testability, complexity, comprehensibility and readability.
2. Researchers then developed a theory of (software) quality by grouping the attributes into four distinct categories: Process quality, Code quality, System quality and Product quality, each influencing the next.
3. The research found two valuable insights: 1] Developers when tasked with improving quality, are more likely to focus on process and code quality to help make the code easier to maintain. 2] Process and Code quality metrics are tracking indicators of overall system quality. Better process and code quality metrics could help organisations gauge where engineering teams should focus quality improvement efforts.
Abi Noda from Engineering Enablement newsletter recently shared a study from Google on Software Quality. Specifically, Google researchers wanted to understand what developers thought about quality and how it affected their productivity. Abi has a good summary, which is well worth a read.
However, what caught my eye were the quality attributes developers seem to care about, which I will cover in this post and how Quality Engineers could use this research, which I will cover in the next issue.
First, a quick recap on what quality attributes are. From What is Quality Engineering?
By thinking of quality as different lenses held by different stakeholders of our products, we can start to build a list of distinctive attributes that they value. From this list, we can define what quality means for our teams and identify which attributes are negotiable and which are not.
Quality can therefore be summarised as follows:
Quality - attributes that are valuable to key stakeholders
Based on interviews and prior studies, the Google researchers identified the code quality attributes developers regularly discussed. These being:
Code quality attributes
Defect rate
Reliability
Maintainability
Testability
Complexity
Comprehensibility (clarity of overall purpose/structure)
Readability (clarity at line/method level).
Where the research gets interesting is how they think about these attributes and the Theory of (Software) Quality that they posited.
Google Researchers Theory of (Software) Quality
The researchers categorised the code quality attributes that developers valued into four distinct categories, with each one influencing the next:
The four categories are:
Process Quality
Code Quality
System Quality
Product Quality
Process Quality
Process quality is what happens before code is written and covers attributes like:
Comprehensive and deterministic testing,
Thorough code reviews
Organisational consistency,
Effective planning process.
There is good evidence that process quality can lead to better overall software quality and is more predictive of post-release defects than existing code quality metrics. The researchers believe that process quality can lead to higher code quality.
Process quality matches my theory of building quality in, specifically, building quality into the process. From What is quality engineering?
When we think about quality, we usually only consider the product quality attributes. However, those attributes are highly influenced by the processes adopted by the team and their inter-team interactions. Therefore, if the quality of their processes or inter-team interactions is poor, these will likely transfer to their products.
This supports my view that building quality into the teams' processes will lead to better-quality products.
Where Google's Theory of (Software) Quality differs to my view of building quality in is that the researchers break down product quality into three distinct areas of Code, System and Product quality.
Code Quality
Code quality is about how easily a developer can understand and change the code, focusing exclusively on the maintainability of the code base. The types of code quality attributes they think about are all subcategories of maintainability:
Testability
Readability
Comprehensibility
Complexity
Moving fast and not breaking things
When developers say code quality is high, then what they are referring to is that the code has a clear purpose and is comprehensible. High code quality leads to high levels of maintainability, which developers believe will increase their velocity. We could interpret this as developers can move fast and not break things. But what they don't mean with high code quality is that the code is reliable or defect-free. Developers recognise that there are other systems that their code interacts with, which is outside of their control and could affect these attributes.
System Quality
System quality is focused on the software system as a whole and will include integration with 3rd party systems. The types of quality attributes in developer's minds here are:
High reliability
High performance
Low defect rates
High security
Good privacy
The researchers shared that system quality is one of the hardest to measure due to the limited data available in teams. For instance, system outages are rare, so if a system has two outages one year and none for the next, we can't say if it was due to the team's actions or luck that prevented the outage.
The researchers found that if code quality was high, there was a good chance that system quality would be, too. Still, they also pointed out that code quality was not the only factor that could affect system quality.
Product Quality
Product quality is primarily experienced by customers or end users of the system. Developer identified the following quality attributes that affect product quality:
Utility
Usability
Reliability
While developers felt they directly influenced reliability, they said they worked with product managers and designers to influence the utility and usability of the products and services.
Developers felt they influenced the product quality in two ways. Low code quality (or maintainability) would slow their velocity, slowing bug fixes or feature requests. They also felt that low code quality would likely lead to higher defect rates, affecting end-user perceptions of product quality.
Developer focus on Process and Code Quality
The researcher found that developers focus on process and code quality attributes when they think about software quality. Therefore, when organisations ask to improve quality, developers are more likely to improve process and code quality. But this is where a mismatch between the organisation and engineering teams can occur.
When the organisation thinks about quality, they are focused on systems and product quality. This can lead to confusion when the developers improve the maintainability of the code, but what was wanted was improved reliability or lower defect rates.
Process and code quality metrics could be helpful for the organisation because they are tracking indicators of system quality. These metrics could give an insight into the quality of the system, which we have the least amount of data on. However, researchers pointed out that existing code quality metrics (cyclomatic complexity) are not predictive of system quality. This is something to keep in mind if you plan to use code metrics to predict system quality.
A theory of software quality
I've been very particular about appending software to Google's Theory of Quality. This is because the researchers focused quite heavily on software developer's views when developing their theory. Which, understandably, puts a heavy slant towards designing, writing and maintaining code. Because of this narrowed view of quality, this theory is more about software quality than a quality of general software systems. A general quality of software systems would also encompass more of the socio-technical system thinking within which the software is created.
Conclusion
Google's research into developers' views on quality is valuable in helping shed light on a key stakeholder's view of system quality. Through interviews and literature studies, they identified several quality attributes developers think about when considering quality. Categorising those attributes into the process, code, systems, and product quality gives engineers a framework of how to think about quality and how they can influence other system attributes.
Quality Engineers can use this information to further their understanding of how quality is created, maintained and lost. In the next post, I plan to look at how quality engineers can use this research and where I think more research could be done to expand the theory further.
This is a free post 🔓 so if you find it helpful, feel free to forward it on as others may do too, and if you’re not already a subscriber, then sign up so you can stay up-to-date with the latest Quality Engineering posts.