The Myth of “Bug Free” Software

First things first, let’s just get this out of the way now and make sure the correct expectations are set:

There is no such thing as bug free software.

I can’t count the number of times a client has asked me about this, nor can I count the number of times I have had to give a client that answer. There is simply no such thing as bug free software.

I have heard, and read, all manner of chicanery and hoodwinkery related to “this process” or “that methodology” which, when followed correctly, miraculously result in zero bugs present. I have yet to actually see any of these claims concretely implemented (in a piece of software, a website or a mobile app) with any kind of real world complexity and functionality. This naturally leads to an immediate follow-up question:

How do you determine when the software is ready to launch?

It’s a good question and it bears some thought, because the answer is not a simple one. First, there’s the tester’s feedback to take into consideration, such as: How solid or shaky is the software? Then there’s the question of coverage. How much of the core functionality has been covered and to what depth? What about new features, if any? How thoroughly have those been tested?

In addition, you can’t forget about the regression testing: Checking that the bugs that have been marked fixed are not only fixed, but that those fixes have not spawned a bevy of new bugs. Then there’s the metrics to consider.

Review the Metrics

Metrics can be helpful, but by no means can they be taken alone as the stamp of approval to push a release build into the wild. Certain trends, however, should be evident prior to launch. The rate of bug discovery, for example, should be trending down.

There will naturally be spikes and these are typically caused by a fix which results in unforeseen issues or by the tester exploring a new area of the software. But overall, this should be trending down.

The number of bugs fixed typically spikes and then tapers off as minor and trivial issues are deferred to the next build or are closed and pushed to the FAQs. Both the number of high severity issues as well as the rate of discovery for those issues should be trending down.

Look at the Whole Picture

All of this data, metrics, tester feedback, coverage and regression status, when taken together, help to determine when the software is ready to launch. Does this mean that the software is bug free? No, there are always more bugs. It does however mean that the software has been tested as thoroughly as possible within the time constraints of the project, with the team that you have available.

There is no such thing as bug free software. However, you can minimize the volume and the severity of the bugs present, as well their impact on your users, through rigorous testing.