Wednesday, September 21, 2011

Best Practices For Regression Testing

SDLC defines that when a defect is fixed, two forms of testing are to be done on the fixed code. The first is confirmation testing to verify that the fix has actually fixed the defect and the second is a regression test to ensure that the fix itself, hasn't broken any existing functionality. It is important to note that the same principle applies when a new feature or functionality is added to the existing application. In the case of new functionality being added, tests can verify that the new features work as per the requirement and design specifications while regression testing can show that the new code hasn't broken any existing functionality.

It is possible that a new version of the application will have fixed previously reported defects as well as having new functionality. For the 'fixes' we would normally have a set of  Test Scripts (Test cases) which are run to confirm the fixes, while for the new functionalities we would have a set of  Functionality test cases.

Overtime, as the software application becomes bigger and bigger in terms of new functionality and more components are added, a regression pack, which is a bank of test cases, is developed to be run on new version of the application which is to be released.

Selecting tests for regression packs

As explained earlier, for each new release of software application, three sets of test suites are executed; Regression Tests, Release Specific Tests and Defect Test Scripts. Choosing test cases for regression packs is not a trivial exercise. Careful thoughts and attention need to be paid on choosing the sets of tests to include in the regression packs.

One would expect that as each new test case written for Release Specific Tests, they will become part of the regression pack to be executed after the next version of the code is arrived. So, in other words, the regression pack becomes bigger and bigger as more and more new versions of the code is developed. If we automate regression testing, this should not be a problem, but for a manual execution of large regression packs, this can cause time constraints and the new functionalities may not be tested due to lack of time.

These regression packs often contain tests that cover the core functionality that will stay the same throughout the evolution of the application. Having said that, some of the old test cases may not be applicable anymore as those functionalities may have been removed and replaced by new functionality. Therefore, the regression test packs need to be updated regularly to reflect changes to the application.

The regression packs are a combination of scripted tests that have been derived from the requirement specifications for previous versions of the software as well as random or ad-hoc tests. A regression test pack should, at a minimum, cover the basic workflow of typical use case scenarios. "Most Important Tests" i.e. tests which are important to the application domain should always be included in the regression packs. Successful test cases, i.e. tests which have revealed defects in the previous versions of the application are also a good candidate to be included in the regression packs.

Search in this page

References: Some of the contents may have reference to various sources available on the web.
Logos, images and trademarks are the properties of their respective organizations.