Wednesday, 16 May 2012

Why your software has bugs

I really like getting things right. Getting things right was such a rewarding skill at school that I've never managed to unlearn the lesson. And that's unfortunate, because writing software isn't about getting things right.

Writing software, like making any tool, is about getting things right enough.

The value of software is a function of the amount of work you can automate. The more your software can correctly do things on your behalf, the more time you can reclaim in which to make money in some additional way. So, if we (simplistically) define quality of software as the fraction of the time that it does the right thing¸ then its value should be a linear function of its quality. The line won't start at zero, because software can do the wrong thing very very fast and then you have to clear up after it. And it will never reach 100%, because even in the best case you'll have to spend a second or two casting an eye over the results to make sure things did go according to plan.

The cost of software doesn't work like that, though. Doing twice as much testing doesn't find you twice as many bugs, and there will always be some defects lurking in obscure corners where nobody has thought to look.

We can sketch the cost and value of software:

It's clear that, beyond a certain quality threshold, the cost of producing the software will outweigh the value that can be derived from it. Make the product that little bit more perfect, and it will have cost more to make than you can possibly hope to gain from it.

But it's worse than that. At the intersection of those two lines, the author of the software is spending all available resources on making software of the highest possible quality, making zero gain. It's better to maximise the difference between the two curves. The optimal quality is further down the scale:

This turns out to be a result in production theory and it's easy to find equivalents to it in all sorts of other fields. And the consequence is equally general. Once that blue curve is headed back downwards, every incremental increase in quality - every bug fixed - costs more than it's worth. There really is no point in fixing those bugs.