How much testing is enough? June 13, 2015 / ToanN / 8 Comments Can you guys share your ideas? Sharing is caring:Click to share on Facebook (Opens in new window)Click to share on Twitter (Opens in new window)Click to share on LinkedIn (Opens in new window) Question Previous post Software Testing Humor and Jokes that Brighten Your Day Next post How to call external REST API but I don't want to trigger to services provider? 8 Comments Thanh Huynh June 14, 2015 at 12:14 pm @Toan, It’s a tricky question. I find testing is enough when: * Team agree on current testings performed and their results * Release date is more important than more tests to perform * Team is well-informed on testing status * Testing budget is running out As a testers, I always to test more. There are always more cases to test. However, as we all know, we don’t have all the time in the world to do the testing. We need to balance when to test and when to stop. Thanks again for your questions. ToanN (Post author) June 15, 2015 at 9:30 am Hi Thanh, On my opinion, before testing, we have to decide pass rate to make sure how many percentage for any function. Ex: 80% passed case, 20% failed case or… Also, as you mention: * Budget of project * Resource * About date line: it’s belong to progress management, we can’t say we are nearly deadline but still have a lot of things can’t finish… Thanks your ideas, Thanh Huynh June 15, 2015 at 9:55 am @Toan, Re: “Ex: 80% passed case, 20% failed case or..” > Sure, you can rely on these exit criteria to exit the testing or say we do enough testing. If you take a further steps, try to understand why these numbers come from. To me, it’s all about the answer to the question “are we confident enough to release this software now?” Basically, I recommend communication among team over numbers. Happy testing! ToanN (Post author) June 15, 2015 at 10:13 am Hi Thanh, Can you tell me your idea about “why there numbers come from”? Basically, I recommend communication among team over numbers. => It’s not only communication, but also it’s on planning (project management process), with customer will decide base on traceability matrix for this function to make it release. Thanh Huynh June 15, 2015 at 11:28 am @Toan, Sorry, it’s a typo, I mean “where these numbers come from” not why… I do agree the exit criteria is a planning work but in order to make the plan works, a proper communication is needed. As far as experience, plan is likely to be out of date at some point of time. That’s why I recommend constant communication over plan Paul Seaman June 20, 2015 at 4:46 pm ToanN “On my opinion, before testing, we have to decide pass rate to make sure how many percentage for any function. Ex: 80% passed case, 20% failed case or…” Let’s say before testing you agree on 80% passed test cases. You finish testing and achieve 85%. So is testing now all OK? What if that 15% of fails are issues that would prevent the software going in to production? Does that agreed metric really tell you anything useful about the software? Are there other things you could consider instead? Thanh Huynh June 22, 2015 at 4:58 pm Re: “Let’s say before testing you agree on 80% passed test cases. You finish testing and achieve 85%. So is testing now all OK? What if that 15% of fails are issues that would prevent the software going in to production?” Good points Paul. I wish more managers could be aware of the dark side of metrics before assigning the numbers to team. Tri Nguyen July 4, 2015 at 8:46 pm I think the exit criteria based on pass rate is not enough. It also must based on how much percent of critical bug/issue and how they effect on other functions especially on main function. That also should be reported to customer and if they are agree then we have a good signal to go. Cheers! 🙂 Leave a Reply Cancel replyYour email address will not be published.Comment Name Email Website Save my name, email, and website in this browser for the next time I comment.