Be careful what you optimise for

This week’s links remind me of a topic I think about often:
What is your system really optimising for?

Is it speed, quality, or both?

Because quality is not created by intention alone. It is shaped by the conditions around the work. For instance, the feedback loops we have, the incentives we create, the trade-offs, the things we measure and the things we choose not to see.

This week’s links look at that from a few different angles: culture of quality, joy in engineering, AI-generated work, reward systems, productivity measures, and why technical excellence still needs to be connected to customer value.

---

🎧 Prefer to listen? There’s also an audio version of this Linky. It’s not just a reading of the post. I talk through the links, why they stood out to me, and what I’m taking from them from a quality engineering perspective.

---

Latest post from the QE Newsletter

[REPOST] Three approaches to foster a culture of quality

This week I reposted my creating a Culture of Quality post, which focuses on three approaches organisations can use to build quality in.

A key takeaway from this post is that a lot of conversations about quality tend to focus on roles:

• Do we need testers?

• Can developers test reliably?

• Should we have quality engineers?

And those are useful questions, but they can also become a distraction. Because quality is not created by roles alone. It’s created by how your system behaves.

What the roles do, though, is influence that behaviour through how people think, act, collaborate, share information, make trade-offs, and respond to risk. But changing job titles or shifting responsibilities does not automatically improve quality.

Because if the system still rewards speed over learning, hides problems until the end and treats testing as a gate, then the quality problems will remain.

That is why I think a culture of quality is less about saying “quality matters” and more about designing the conditions in which quality can actually emerge through the work.

---

The joy of software engineering

From Gene Kim’s “Five Ideals” from The Unicorn Project for rescuing broken software, which Dave Farley has mapped onto five CI/CD engineering practices. The one I wanted to call out was joy:

2. “Joy” is not a word engineers use at work very often, but there is certainly no joy, and no flow, in 45-minute builds, flaky tests, and constant friction. Software development is a flow-based discipline optimised for learning. We use Continuous Delivery, fast tests, small batch sizes, and reliable pipelines to systematically remove friction so developers can actually focus.

Joy is not something we talk about much in software, but I think it matters more than we realise.

Having moments of joy in your work helps keep you motivated and pushing forward. Even more so when things are difficult, because it is often in those tough moments that the real learning happens.

But if the work is constantly painful, slow, frustrating, and full of avoidable friction, people begin to dread it. They stop exploring, improving things and generally stop caring as much as they could.

Joy is not about pretending work should always be fun. It is about creating the conditions in which people can do good work, see progress, learn from feedback, and not have to fight the system every day. Via The five ideals of software projects | Dave Farley

If AI automates all the easier parts, who has to handle the hard bits?

The hard stuff that remains is the most difficult work humans have to do. Staying vigilant for rare/odd/weird problems over a long duration

And that’s because AI produces stuff that looks “done” but isn’t in a multitude of subtle, almost invisible ways that are difficult to predict ahead of time

And that characteristic isn’t going to disappear, it’s inherent in the technology

The trouble is, the ability to spot those weird problems is improved by how involved you were in building the thing in the first place

But AI is doing the building...

...so we’re eroding our ability to spot problems the more we use it

This feels like one of the biggest causes of issues we might see with AI.

The more we offload cognitive tasks to AI, the less able we may be to intervene when it encounters something it cannot handle.

That does not mean we should avoid AI. But it does mean we need to think carefully about where understanding is being built, where it is being lost, and what skills we still need people to practise.

Some expertise is still going to be needed. Probably more than we think. But how people maintain that expertise as AI does more of the building will be interesting.

As Vernon said, our ability to spot the subtle, rare, weird problems often comes from being close to the work. If AI creates distance between people and the work, then we need new ways to keep people close to the reasoning, the trade-offs, and the failure modes.

Otherwise, we may increase output while quietly reducing our ability to understand what we have created. Via Here are some interesting things that have crossed my path recently | Vernon Richards

Be careful what you reward

It is an old paper from 1995, but the examples shared really help show how often we hope for one thing but reward something else.

The causes are usually things like:

• objectives that only work in controllable, routine environments, like factories

• measuring what is easy and visible over what is actually desired

• pretending we want one thing publicly while quietly rewarding another thing in practice

This feels very relevant to quality engineering because all organisations say they want quality. But they may reward speed, output, utilisation, certainty, or short-term delivery over learning, maintainability, risk reduction, and customer outcomes.

That doesn’t make people bad, but it does make the system in which they work powerful. People tend to respond to what the system rewards, especially when the pressure is high.

Which makes me wonder whether AI will make this easier or harder.

On one hand, AI could help us process large amounts of verbatim feedback and make more of the invisible visible. We might be able to better understand patterns in customer pain, engineering friction, incidents, customer reviews, and support requests.

But on the other hand, AI could also make it easier to produce the appearance of evidence. It could help generate the narrative we want to show, while the system underneath continues to reward something else.

This is why incentives are so dangerous. They don’t just measure behaviour, they help shape it. Via After reading this article on Meta employees incentivized to burn 2 trillion tokens a day | Ethan Mollick

Coding throughput as a measure of productivity

TestPappy has been reading the ThoughtWorks Tech Radar, and there are a few cautions with AI in there:

The warning is simple. If you measure AI success by lines of code generated or pull requests opened, you will get exactly that. A flood of it. And not much else. It’s the age-old metric gamification, and obviously AIs are good at it as well.

This is the same point as the reward paper: measuring something easy to see rather than what we actually want.

Things like:

• Lines of code generated.

• Pull requests opened.

• Tickets moved.

• Velocity increased.

These things might look like productivity, but they do not necessarily tell us whether we are getting better, faster outcomes powered by software.

So what should we measure?

ThoughtWorks suggests the first-pass acceptance rate, or how often AI output can be used with minimal rework.

I like this because it starts to expose the hidden work, things like long reviews, reworking, the back and forth of reviews and fixes.

If AI-generated PRs are constantly having to be reworked, that is useful feedback. It might suggest that teams need more support with prompting, better AI-priming documents, clearer engineering standards, smaller slices of work, or different ways of using the tools.

We shouldn’t focus on more AI output, but on improving the quality of the interactions among people, tools, code, and feedback loops.

There is one risk, though: people may just accept the output and not bother reviewing it at all 😱, which brings us back to the system again. Any measure will be gamed if the surrounding incentives are wrong. Via The Tech Radar is Blinking Red | Test Pappy

The Principal Engineer’s Path: Skills, Strategies, and Lessons Learned

Great talk by Sophie Weston on her 30-year engineering career and operating as a principal engineer.

She covers:

• Leadership: how moving forward and backwards in your career can be helpful

• Personal development: as you gain more experience, you have to own your development and not expect your manager to guide you at every step

• Networking: building relationships inside and outside your organisation is just as important as learning the tech

• Pushing your comfort zone: speaking, writing, and developing yourself in areas where you have less skill really helps long-term

• Helping others: people will have helped you get where you are, so you should pay it forward and help others when they are trying to get better

The whole talk is worth a watch and echoes a lot of my own career, especially around personal development, networking, and helping others.

The reason I think this connects to quality engineering is that so much of quality work happens through influence.

Yes, technical skill matters. But as QEs, our work is often about creating better conditions for others to make good decisions.

That means helping teams see risks earlier, connecting people across boundaries, sharing context, building trust, making the invisible work visible and helping others develop their judgement.

Sophie’s recommended reading list is also well worth a look, and I can highly recommend many of the books on it.

Via The Principal Engineer’s Path: Skills, Strategies, and Lessons Learned | InfoQ

Building quality in does not matter if you are building the wrong thing

A cautionary tale from James McGivern from when he was a tech lead in a team.

They had built a high-performing system, but...

…the numbers that actually mattered were moving in the wrong direction. Customer acquisition costs were climbing. New customer acquisition rates were falling. Onboarding took weeks, sometimes months – not days. These weren’t metrics I felt accountable for. They weren’t on my dashboard. So I didn’t see them as my problem to solve.

The product model was broken. Initial costs for customers were too high; the return too uncertain. We needed to cheaply explore the problem space - to find where our technology could actually solve problems customers would pay to have solved. Instead, we doubled down on delivery. We made the engine more powerful without asking whether we were driving in the right direction.

This is why knowing what your customers value, based on actual data, is so important. Because it does not matter how well you build the product if it is not the right thing to build.

But I also do not think that means “building it right” should be ignored - it’s a balance.

When you are trying to find product-market fit, the balance may tip more towards speed, exploration, and cheap learning. Once you have found something valuable, the balance may shift more towards reliability, resilience, maintainability, and scale.

As quality engineers, we should care about the feedback loops across all of those stages.

It’s not just about the code working, fast pipelines and getting the release out. But whether the thing we are building is creating the outcome we hoped for.

That does not mean every team needs every measure from day one. But it does mean we should be thinking about what feedback we need, when we need it, and what decisions it helps us make.

Because building quality in is not only about engineering the thing well. It is also about learning whether the thing is worth building in the first place. Via When Technical Excellence Isn’t Enough: A Cautionary Tale | James McGivern

---

As always, I’d love to know what you think. You can reply to directly to one of my emails, leave a comment, or message me on socials.

And if this post made you think of a team conversation you need to have, send it to someone and use it as a starting point. Maybe ask them: what are we actually optimising for right now?

---

Past Linkys