It’s a tricky question. I find testing is enough when:
* Team agree on current testings performed and their results
* Release date is more important than more tests to perform
* Team is well-informed on testing status
* Testing budget is running out
As a testers, I always to test more. There are always more cases to test. However, as we all know, we don’t have all the time in the world to do the testing. We need to balance when to test and when to stop.
Thanks again for your questions.
ToanN
Hi Thanh,
On my opinion, before testing, we have to decide pass rate to make sure how many percentage for any function. Ex: 80% passed case, 20% failed case or…
Also, as you mention:
* Budget of project
* Resource
* About date line: it’s belong to progress management, we can’t say we are nearly deadline but still have a lot of things can’t finish…
Re: “Ex: 80% passed case, 20% failed case or..”
> Sure, you can rely on these exit criteria to exit the testing or say we do enough testing. If you take a further steps, try to understand why these numbers come from. To me, it’s all about the answer to the question “are we confident enough to release this software now?”
Basically, I recommend communication among team over numbers.
Happy testing!
ToanN
Hi Thanh,
Can you tell me your idea about “why there numbers come from”?
Basically, I recommend communication among team over numbers.
=> It’s not only communication, but also it’s on planning (project management process), with customer will decide base on traceability matrix for this function to make it release.
Sorry, it’s a typo, I mean “where these numbers come from” not why…
I do agree the exit criteria is a planning work but in order to make the plan works, a proper communication is needed. As far as experience, plan is likely to be out of date at some point of time. That’s why I recommend constant communication over plan
Paul Seaman
ToanN
“On my opinion, before testing, we have to decide pass rate to make sure how many percentage for any function. Ex: 80% passed case, 20% failed case or…”
Let’s say before testing you agree on 80% passed test cases. You finish testing and achieve 85%. So is testing now all OK? What if that 15% of fails are issues that would prevent the software going in to production?
Does that agreed metric really tell you anything useful about the software? Are there other things you could consider instead?
Re: “Let’s say before testing you agree on 80% passed test cases. You finish testing and achieve 85%. So is testing now all OK? What if that 15% of fails are issues that would prevent the software going in to production?”
Good points Paul. I wish more managers could be aware of the dark side of metrics before assigning the numbers to team.
Tri Nguyen
I think the exit criteria based on pass rate is not enough. It also must based on how much percent of critical bug/issue and how they effect on other functions especially on main function. That also should be reported to customer and if they are agree then we have a good signal to go.
@Toan,
It’s a tricky question. I find testing is enough when:
* Team agree on current testings performed and their results
* Release date is more important than more tests to perform
* Team is well-informed on testing status
* Testing budget is running out
As a testers, I always to test more. There are always more cases to test. However, as we all know, we don’t have all the time in the world to do the testing. We need to balance when to test and when to stop.
Thanks again for your questions.
Hi Thanh,
On my opinion, before testing, we have to decide pass rate to make sure how many percentage for any function. Ex: 80% passed case, 20% failed case or…
Also, as you mention:
* Budget of project
* Resource
* About date line: it’s belong to progress management, we can’t say we are nearly deadline but still have a lot of things can’t finish…
Thanks your ideas,
@Toan,
Re: “Ex: 80% passed case, 20% failed case or..”
> Sure, you can rely on these exit criteria to exit the testing or say we do enough testing. If you take a further steps, try to understand why these numbers come from. To me, it’s all about the answer to the question “are we confident enough to release this software now?”
Basically, I recommend communication among team over numbers.
Happy testing!
Hi Thanh,
Can you tell me your idea about “why there numbers come from”?
Basically, I recommend communication among team over numbers.
=> It’s not only communication, but also it’s on planning (project management process), with customer will decide base on traceability matrix for this function to make it release.
@Toan,
Sorry, it’s a typo, I mean “where these numbers come from” not why…
I do agree the exit criteria is a planning work but in order to make the plan works, a proper communication is needed. As far as experience, plan is likely to be out of date at some point of time. That’s why I recommend constant communication over plan
ToanN
“On my opinion, before testing, we have to decide pass rate to make sure how many percentage for any function. Ex: 80% passed case, 20% failed case or…”
Let’s say before testing you agree on 80% passed test cases. You finish testing and achieve 85%. So is testing now all OK? What if that 15% of fails are issues that would prevent the software going in to production?
Does that agreed metric really tell you anything useful about the software? Are there other things you could consider instead?
Re: “Let’s say before testing you agree on 80% passed test cases. You finish testing and achieve 85%. So is testing now all OK? What if that 15% of fails are issues that would prevent the software going in to production?”
Good points Paul. I wish more managers could be aware of the dark side of metrics before assigning the numbers to team.
I think the exit criteria based on pass rate is not enough. It also must based on how much percent of critical bug/issue and how they effect on other functions especially on main function. That also should be reported to customer and if they are agree then we have a good signal to go.
Cheers! 🙂