Cost Of Defect

I recall from my early consulting days a graphic that was often hauled out at management meetings and retrospectives. It looked something like this:

I’ve often wondered the source of this wisdom and discovered after some research that it comes from Capers Jones, Software Assessments, Benchmarks, and Best Practices, Addison-Wesley, 2000. Yep, Capers calculated this out to the dollar for us; although, in my memory similar graphics have been used at least since 1990.

In pondering the graph, what should one conclude? Search Google, and top results use it to justify the investment of extra time, effort, and capital in fixing errors during the early tasks of requirements, design, and coding. Spend enough in these tasks and a reasonable conclusion is a savings of $2,276,700 on a typical project.

What if, though, we assumed instead of phases on the horizontal axis, it merely represented time? The conclusion would be quite different. If we could complete requirements, design, coding, testing and acceptance in the time typically taken for requirements alone, we would never pay 10 times the starting cost to fix a defect. Instead of saving money, added investment in a single early task almost guarantees higher costs of correcting defects later in the time line.

I’d like to do the scholarly research of Capers around this to get empirical data. In the absence of that, using anecdotal evidence from applying agile to software development for the last eight years, I feel much more comfortable with this model. I’ve been on very long projects where early assumptions proved wrong or changed because different interest groups looked at them later. I’ve experienced long requirements cycles that delayed introduction of software only to find out that key features were missing. With agile, development happens across all phases more quickly and these problems are largely alleviated.

Thus, my second “law of development physics”: The cost of correcting a defect rises exponentially with the time taken to identify the defect. Or more generally, bad news does not age well. (The first law is coming next.)