Software bugs cost economy billions

Software bugs cost the U.S. economy an estimated US$59.5 billion per year, or 0.6 per cent of the gross domestic product, according to a recent study by the U.S. Department of Commerce National Institute of Standards and Technology (NIST).

And it’s quite likely that the Canadian problem is of the same proportion, said several software quality and assurance experts in this country.

“I see that there is an inordinate amount of time spent fixing things that shouldn’t have to be fixed in the first place,” said Edgardo Gonzalez, president of Calgary-based Projects Recovery Specialists Ltd. In fact, Gonzalez said 60 or 70 per cent of his work is going in to organizations to fix inadequate systems development processes.

According to NIST more than half the costs are carried by software users and the remainder by software developers and vendors. Additionally, the study found that while all errors cannot be removed, more than a third of the costs, or an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of defects.

Currently, more than half of errors are not found until “downstream” in the development process or during post-sale use of software, NIST said.

NIST said software is error-ridden in part because of growing complexity. Developers already spend about 80 per cent of development costs on identifying and correcting defects, and yet few products of any type other than software are shipped with so many errors.

Other factors contributing to quality problems include marketing strategies, limited liability by software vendors, and decreasing returns on testing and debugging, the study found.

“The industry gives a tremendous amount of lip service to fixing bugs, but the truth of the matter is that our industry is extremely deadline-driven. If you don’t get your software out there somebody else is going to eat your lunch,” said Wesley Nelson, a Toronto-based senior software architect.

Because consumers often want to commit to the firm that delivers the software first – even if it’s buggy – you wind up with a situation where software vendors operate on the understanding that its more important to lock in the customer than it is to deliver quality software, Nelson said.

Although bugs come from many sources, from incomplete gathering of customer requirements, to overall design, to the quality of coders involved, Nelson said, “it’s really a question of the maturity of the shop and the maturity of their processes from end-to-end.”

The core issue is difficulty in defining and measuring software quality, according to NIST.

To boost software quality, testing needs to be improved, NIST said. Standardized testing tools, suites, scripts, reference data, and implementations and metrics that have undergone a rigorous certification process would have a large impact on inadequacies that now plague software markets, NIST said.

But more testing by itself is not going to do it, Gonzalez said.

“Integrated quality management has not yet been advocated as part of the process. We have to change the way in which people work and projects are structured so that quality is introduced at each stage (instead) of leaving it as an event at the end of the project,” he said.

Projects Recovery Specialists Ltd. is at The full NIST report is at

– With files from IDG News Service.