Register Now

Login

Lost Password

Lost your password? Please enter your email address. You will receive a link and will create a new password via email.

Smoke test vs Sanity test vs retest vs regression test

smoke-test-vs-sanity-test-vs-retest-vs-regression-test

As a software test engineer, you may probably hear about smoke testing, sanity testing, retesting and regression testing. Very likely, many of them are testing activities that you often do on a daily basis. However, for junior testers (and even for experienced testers) distinguishing these terms can be a real challenge.

In many cases, you can still do the job right even if you can’t tell the difference. However, as a professional tester, knowing what you do and why you do things is always the right thing to do.

In this post today, I would like to help you clarify the difference between smoke testing, sanity testing, retesting and regression testing to clear all the confusion you may have…and move forward.

Alright, let’s do it.

I’ll go into details right below, but before that, here’s a quick definition of each terms:

  • Smoke Testing: We do smoke test whenever we receive a new build that is relative unstable to ensure that the critical functionalities of the AUT working fine. The idea of this test is to find critical problems as soon as possible and reject the build so that we don’t waste time to test further (the broken build).
  • Sanity Testing: We do sanity test whenever we receive a new build that is relative stable to conduct to identify the dependent missing functionalities. In other words, you will validate the major functionalities of the system in a more detailed way.

Both sanity tests and smoke tests are ways to avoid wasting time and effort by quickly determining whether an application is too flawed to merit any rigorous testing.

  • Retesting: We do retesting for a feature/function which had defects (bugs) & those bugs have been fixed recently.
  • Regression Testing: We do regression testing for AUT when there are changes including new features / bug fixed to make sure no impact to the current existing functionalities.

To have better understanding, let me introduce you the table of comparison as below:

Smoke TestingSanity TestingRegression TestingRe-testing
Smoke testing is executed to determine if critical functionalities of AUT are working fineSanity testing is executed to determine if the section of AUT is still working as design after some minor changes or bug fixesRegression testing is executed to confirm whether a recent program or code change has not adversely affected existing featuresRetesting is executed to confirm the test cases that failed in the final execution are passing after the defects are fixed
The purpose of smoke testing is to verify the "stability" of the system in order to proceed with more rigorous testingThe purpose of smoke testing is to verify the "rationality" of the system in order to proceed with more rigorous testingThe purpose of regression testing is that new changes should not have any side effects to existing functionalitiesRe-testing is done on the basis of the defect fixes
Defect verification is not the part of smoke testingDefect verification is not the part of sanity testingDefect verification is not the part of regression testingDefect verification is the part of re-testing
Smoke testing is executed before regression testSanity testing is executed before regression testing, after smoke testingBased on the project and availability of resources, regression testing can be carried out parallel with Re-testing- Re-testing is done before we start sanity testing
- Priority of re-testing is also higher than regression testing, so it is carried out before regression testing
Smoke test can be excuted manually or automaticallySanity test often is executed manuallyYou can do automation for regression testing, manual testing could be expensive and time consumingYou cannot automate the test cases for re-testing
Smoke testing is subset of Regression testingSanity testing is subset of Acceptance testingRegression testing is only done when there is any modification or changes become mandatory in existing projectRe-testing executes a defect with same data and the same environment with different inputs with new build
Test cases of smoke test are part of regression testing, only cover critical functionalitiesSanity Test can be performed without test cases but domain knowledge is requiredTest cases for regression testing can be obtained from the functional specification, user tutorials and manuals, and defect reports in regards to corrected problemsTest cases for re-testing cannot be obtained before start testing.

Wrap-up

Hopefully after reading this article, you will have the basic knowledge of smoke testing, sanity testing, retesting and regression testing. Like I said at the beginning, it’s ok if you can’t tell the difference among those terms. Terminologies are confusing all the time. However, by knowing what you do and call it by its right name is how a professional tester should do.

Like it? Don’t like it? I would love to hear your thought.

Comments ( 2 )

  1. Tai
    January 3, 2017 at 2:17 pm
    Hi aLong Manh,
    Thank you for sharing your knowledge about testing. But after read your article, something is seemly that not clear with me. So could you please give me example about type of this test in basic function? I think it will clearer than just only theory.
    Hope that will have your reply soon :)
  2. Long Manh
    January 8, 2017 at 11:44 pm
    Hi Tai,

    Firstly, I want to thank you for your interest in this topic!

    To answer your question, I will give some example to make it easier to understand:

    - Retesting: also is called bug verification. That means when a bug is fixed, we testers have to re-verify the bug to see if it is actually fix or not, by following the old steps, or the new steps that may reproduce the bug.
    Example: you found BUG-001. Now BUG-001 is fixed, you have to check it. This action is called retesting or bug verification.

    - Regression test: running a set of test cases that covers all functionalities of the AUT. After so many changes are made into the AUT, we may have to run the test cases to ensure the whole system is ok. At that time, we have to run the set of test cases for some or all functionalities.
    Example: Yahoo! has many functions that are Login, Logout, Register New Account, Create New Email, Send Emails, Delete Emails, etc. The test cases for these functions are around 4000. Now we have to run these test cases to ensure that Yahoo! system is stable. Depending on the situation, we may run test cases for Create, Send and Delete Emails. Or We will have to run all of the 4000 test cases. That's called Regression Test

    - Smoke test: running a set of test cases of critical functionalities of AUT, which ensures that the major functionalities of the application are working fine.
    Example: Yahoo! has some critical functions that are Login, Logout, Register New Account, Create New Email, Send Emails, Delete Emails. And the test cases for these function are just a few for each functions.

    - Sanity test: is performed when development team needs to know quick state of the product after they have done changes in the code or there is some controlled code change in a feature to fix any critical issue, and stringent release time-frame does not allow complete regression testing.
    Example: in a project there are five modules like login page, home page, user detail page, new user creation, and task creation etc. So we have the bug in login page like on login page username field accepts the less than six alpha-numeric characters which are against the requirements as in requirements it is specified that username should not be below than six characters but as username accepts the less than six characters it is the bug.
    So now the bug is reported by the testing team to the developer team to fix it. When the developing team fixes the bug and passed it to testing team than the testing team checks the other modules of the application means checks that fix bug does not affect the functionality of the other modules but keep one point always in mind that testing team only checks the extreme functionality of the modules, do not go deep to test the details because of the short time so this is the sanity testing.

    Hope you find my answer useful!

    Best regards,
    LM

Leave a reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

About [Long Manh]

Long is content writer at AskTester. Long has 7 years of experience in software testing and he loves messing around.

I Help Testers Do Better Testing

  • + Subscribe to receive software testing tips
  • + Download eBook: My Best Online Resources to Learn Software Testing
  • + Download eBook: A Complete Guide to Becoming a Software Tester