AI and the new economics of quality

Let's start with an analogy, because everybody loves that. You come to your kitchen and find a puddle of water forming around the sink. You call a plumber, takes a look, and gives you the verdict that you were dreading:

"Your draining pipes are done for, you'd have to rip it all out and rebuild from scratch. I'd have to check properly but I'm estimating that it's going to be about $2,000 to fix"

"Damn, replacing my entire drainage system wasn't in my bingo card this morning. Is that the only option? can't we just patch it?"

"Well, you could, but it wouldn't be a permanent solution, most likely it will burst at some other point in a year or so.

"And when that happens?" "When that happens we're back to where we are now, and you need to consider the full rip-and-remake or another patch that will buy you maybe another year"

"How much would the patch cost?"

"About $500"

So you do the math. Permanent solution for $2,000 or patch that you'll have to repeat approximately every year for $500. The permanent solution sounds pricey, but after four years you'd break even and from the fifth year onward you'd be saving money. Many would say that this is a no-brainer, but there are other factors to consider: maybe you just don't have $2,000 to spare right now, so it makes sense to do the patch now and then plan for the big cost next year. Or maybe you're planning to sell your home in one or two years and you decide that you can patch now and leave the permanent solution for the new owner (questionable ethics, but each to their own). Either way, although logic would dictate that one option is better than the other, there are always reasons for argument.

Enter the new technology

Let's repeat this scenario in a parallel world, one where plumbers have developed some amazing patching technology that makes the process really straightforward and cheap. Now the prices change. Rip-and-remake still costs $2,000, but the temporary solution is just $100. That would still buy you just a year, but now we're saying that the "proper" solution has a 20 year break-even point with regards to the annual patches. Now the math ain't mathing any more. 20 years is a long time to wait for an investment to yield benefits. A lot of things could happen in 20 years: maybe the patching technology improves and they last more than a year each time. Maybe the rip-and-remake option also finds some technical advances that make it cheaper in the future. Or maybe you're planning to sell at some point in the next 10-15 years and you decide that this is a next owner's problem (always a pleasant surprise as a buyer).

It may seem that now the no-brainer has switched sides, but we still have other things to consider. Maybe you don't want the hassle of having to worry about a dripping sink every year. Maybe you're concerned about the water damage that this could cause to furniture or other goods, making the overall repairs more expensive. Or maybe this isn't just your home but a place of work, and you're concerned about the reputational damage of a drain that is broken every single year... people remember that, and they will post it on social media.

Mass-produced items become cheaper and cheaper, while well-crafted items retain their price; eventually, the difference isn't justifiable anymore.

The economics of software

For the past few decades, software has been in an upward spiral sustained by quality:

• Software is expensive to build

• To make a profit, you need to sell it at high cost, or you need to scale to large volumes. Or both.

• To sell at high cost you need a high-quality product that justifies the price tag; to scale to large volumes you need a high-quality product that can sustain the volume without collapse

• To build high-quality products, you need top talent

• Top talent is expensive, which makes software expensive

Traditionally, the main cost in software development was the engineers' salaries, which means that the difference in cost between an average product or a high-quality product was correlated with the difference between the salaries of mid-level or top-level engineers. AI has changed this. As I previously argued, for structural reasons AI can only produce code of average quality. This means that, while the cost of producing code of average quality has been dramatically reduced, the cost of producing code of high quality is the same as it was: you can spin up a bunch of agents to produce products of average quality overnight, but the high-quality stuff still requires the same level of dedication.

This is the math ain't mathing moment of software engineering. Suddenly, the break-even point between average and top quality has vastly increased. High-quality code is as expensive as it was, but mediocre code is suddenly a lot cheaper, which makes high-quality code comparatively more expensive.

Is this the end of high-quality code?

I don't think so. It is true that, for many industries, the economic rationale for high-quality code won't hold up any more, and they'll switch to lower quality but much cheaper software. This is the plastic moment for software. But there will be other industries where they simply can't afford to reduce or even look like they're reducing quality. A social media company may be able to deal with a privacy leak due to a bug caused by AI through some monetary compensation or marketing campaign, and in the equation they probably make more money with more bugs in cheaper software. A power plant can't shrug off a nuclear accident because the programmer who coded the control mechanism was juggling seven different agent workflows and didn't pay enough attention to notice the bug.

And here is the thing that many people miss: liability and trust are not really about controls, they are about narratives. Society, at least today, may accept an error caused by a human if the human can argue that there was a reasonable amount of checks and controls along the way; humans make mistakes even in their best days, that's just life. But an error caused by AI? that's not something that society is ready to accept. People are responsible for thousands of car accidents every day and nobody bats an eyelid, but have one autonomous car cause an accident and it will make the news.

That's why I believe that quality code will not disappear, but it will be confined to those areas where it is truly needed or justified; not necessarily for economic reasons, but for liability concerns.

Aren't you being a little bit dramatic?

It's possible, but taking into account that this same effect has happened to other industries, I'd find it rather shocking that it doesn't happen to software engineering as well. Freakonomics did a very interesting episode where they talked about how the US traded taste and variety of food for abundance and low prices as one not-commonly discussed aspect of the Cold War. Similarly, the fast fashion trend has given us quick style updates at the cost of disposable, low-quality clothing.

All industries try to reduce costs to remain competitive until they become victims of their own success. Software is just about to become the next one.