As a software test engineer, you may probably hear about smoke testing, sanity testing, retesting and regression testing. Very likely, many of them are testing activities that you often do on a daily basis. However, for junior testers (and even for experienced testers) distinguishing these terms can be a real challenge.
In many cases, you can still do the job right even if you can’t tell the difference. However, as a professional tester, knowing what you do and why you do things is always the right thing to do.
In this post today, I would like to help you clarify the difference between smoke testing, sanity testing, retesting and regression testing to clear all the confusion you may have…and move forward.
Alright, let’s do it.
I’ll go into details right below, but before that, here’s a quick definition of each terms:
- Smoke Testing: We do smoke test whenever we receive a new build that is relative unstable to ensure that the critical functionalities of the AUT working fine. The idea of this test is to find critical problems as soon as possible and reject the build so that we don’t waste time to test further (the broken build).
- Sanity Testing: We do sanity test whenever we receive a new build that is relative stable to conduct to identify the dependent missing functionalities. In other words, you will validate the major functionalities of the system in a more detailed way.
Both sanity tests and smoke tests are ways to avoid wasting time and effort by quickly determining whether an application is too flawed to merit any rigorous testing.
- Retesting: We do retesting for a feature/function which had defects (bugs) & those bugs have been fixed recently.
- Regression Testing: We do regression testing for AUT when there are changes including new features / bug fixed to make sure no impact to the current existing functionalities.
To have better understanding, let me introduce you the table of comparison as below:
|Smoke Testing||Sanity Testing||Regression Testing||Re-testing|
|Smoke testing is executed to determine if critical functionalities of AUT are working fine||Sanity testing is executed to determine if the section of AUT is still working as design after some minor changes or bug fixes||Regression testing is executed to confirm whether a recent program or code change has not adversely affected existing features||Retesting is executed to confirm the test cases that failed in the final execution are passing after the defects are fixed|
|The purpose of smoke testing is to verify the "stability" of the system in order to proceed with more rigorous testing||The purpose of sanity testing is to verify the "rationality" of the system in order to proceed with more rigorous testing||The purpose of regression testing is that new changes should not have any side effects to existing functionalities||Re-testing is done on the basis of the defect fixes|
|Defect verification is not the part of smoke testing||Defect verification is not the part of sanity testing||Defect verification is not the part of regression testing||Defect verification is the part of re-testing|
|Smoke testing is executed before regression test||Sanity testing is executed before regression testing, after smoke testing||Based on the project and availability of resources, regression testing can be carried out parallel with Re-testing||- Re-testing is done before we start sanity testing
- Priority of re-testing is also higher than regression testing, so it is carried out before regression testing
|Smoke test can be excuted manually or automatically||Sanity test often is executed manually||You can do automation for regression testing, manual testing could be expensive and time consuming||You cannot automate the test cases for re-testing|
|Smoke testing is subset of Regression testing||Sanity testing is subset of Acceptance testing||Regression testing is only done when there is any modification or changes become mandatory in existing project||Re-testing executes a defect with same data and the same environment with different inputs with new build|
|Test cases of smoke test are part of regression testing, only cover critical functionalities||Sanity Test can be performed without test cases but domain knowledge is required||Test cases for regression testing can be obtained from the functional specification, user tutorials and manuals, and defect reports in regards to corrected problems||Test cases for re-testing can be re-used from previous executed test cases|
Hopefully after reading this article, you will have the basic knowledge of smoke testing, sanity testing, retesting and regression testing. Like I said at the beginning, it’s ok if you can’t tell the difference among those terms. Terminologies are confusing all the time. However, by knowing what you do and call it by its right name is how a professional tester should do.