Traditional Testing

Traditional Software Testing Process - The cash cow demystified

So, I don't need to tell you that most of the software testing in the world is outsourced. I live in Bangalore and I assume a considerable amount of the world's outsourced testing projects are here. I meet testers from most of these organizations and hear the same process of testing software although called by different names their organization executives named them.

Most testers I meet lament about the process (or sucked within it for survival ) and approach they are forced to follow and try talking to their management about better ideas and better ways to test. Those executives don't seem to listen much nor willing to change.

Wondering why they wouldn't?

They have hit a cash cow.

Mark Crowther and I were discussing about Code Coverage & Requirements Based Testing in Test Republic and some how drove me to write about the way most businessmen are fooling their customers through the process they love to follow.

So, here it goes with editing and expansion. Assuming Mark from UK or maybe even you outsourced a testing project to me:

• I would spend a couple of days analyzing your requirement document and bill you for X hours per person involved in my team.
• I would spend a couple of days writing a test plan document ( but not refer to it ) and bill you for 2X hours per person involved in my team for preparing it.
• I would spend a month or two writing test case document ( and refer only to it ) and bill you for 10X hours per person involved in my team for preparing it.
• I would then again create a traceability matrix ( just to fool you and your boss about our coverage ) and bill you for 5X hours per person involved in my team for preparing it.
• So far, total of 18X hours per person involved in the team is the billing.
• Assuming X is 50 hours and there are about 10 members in my team, that's 18 * 50 * 10 = 9000 hours of billing with no single bug found yet. ( Mark wrote about it, too )
  
• If you are paying $20 an hour per person on an average, you would have actually given me a business of $180,000 without me or my team finding any bug yet.
• So after investing $1,80,000 on me and my team, you would want some benefits of that. So, you wouldn't pull the project out or move it to another vendor because more or less he would do the same and you would end up paying another $180,000
  
• Then comes the test case execution cycles for our documented 10,000 tests out of a possible hundred million tests
  
• For every new bug that you find out of the releases I make, my team would spend documenting the new test case, getting it reviewed and resulting in slower testing for you and more money for me.
• So assuming running 10,000 tests take 2 weeks for a team of 10 members to execute. Also assuming least 50 cycles of testing, you would have paid me about $140,000 for a coverage whose value might be not worth of the money.
• Of course there is additional documentation of missed test cases and other template filling activities that will be billed to you.
• To fool you further, I would instruct my team to use some expensive license based tools ( what else will I do with the money you are pumping in ) to give you a sense of faster testing ( by foolishly comparing it with human speed of testing ) and call it "Automation Testing". It turns out that these tools could have helped me find bugs that are of not a great value to you but hey, we want to see test case pass more than fail.
• Your coverage isn't improving much because we have converted manual tests to automated tests ( although its not the same test ) but to show you the speed of our tools.
  
• So think about adding another $25,000 and giving you an illusion of an ROI of $100,000 while pumping multiples of $100,000 from your bank account to mine.
  
• The CMM, TMM, Six Sigma, ISTQB scams are built around this eco-system to enable more money flow for hardly any value. Who knows there could be a cut for the people who know all this and yet do it.
  

Why wouldn't a businessman be glad about the traditional approaches to test software?

Are you asking about what happens to the users of the product?

• Lets bother about the users of your product later during our maintenance billing phase. Don't you know SDLC ends in Maintenance Phase? If we do everything right in the previous stages then how do we prove we follow SDLC when there is hardly any work in maintenance phase?
  

While you are reading this, you shouldn't be thinking of this happening only in India but in most parts of the world and even within places like United States and Europe. There are smart businessmen everywhere. At one end they pay us but at the other end use us to make more money. We need money and they need us to make that. Don't make smart businessmen exclusive to India and leave your own country out of it.

I hope those who outsource start pushing for services that doesn't fool them. Who would actually listen to this argument is testers turned businessmen and testers turned outsourcing heads and testers turned business leaders.

Whenever I visit a testing services company and see a testimonial of a customer who talks about the great ROI they got and faster testing, I wonder what a heavy price they paid to believe so.

Exploratory Testing ++ , Context Driven Testing ++ , Rapid Software Testing ++ or else YourMoney --

Don't want to be fooled by outsourcing software testing? One of those who could be of help to you among many folks I know, is myself.