Unit Test Case Selection to Evaluate Changes in Critical Time

Abstract

Defects blocking major business transactions may be found while a software system is in production. Due to the urgency of returning the system to proper operation, these defects are frequently resolved in the production environment itself, under such a restricted deadline that there is not enough time to run the complete set of unit test cases upon the patched version of the software. Declining to run the test case suite allows quicker deployment to production, but also allows for other defects to be introduced into the system. This paper evaluates whether a heuristic search approach can help a decision-maker selecting a subset of a unit test case suite to validate a set of changes made in the production environment. The selected test cases should maximize the coverage of changed features and must be executed under a strict time constraint. We designed and executed an experiment which compared different algorithms and found that heuristic search can provide better solutions to this test case selection problem (though at higher cost) than local or random search when the time restriction is a small fraction of the time required to run the full suite.

Keywords:

• Software Testing
• Genetic Algorithms
• Test Case Selection
• Process development
• Risk Management

The authors would like to express their gratitude for FAPERJ and CNPq, the research agencies that financially supported this project.

Notes

1 http://subversion.tigris.org/

2 http://www.apache.org/

3 http://www.debian.org/

4 http://www.junit.org/

5 http://www.nunit.org/

6 http://sourceforge.net/projects/jcover/

7 http://www.ncover.com/

8 http://www.filehelpers.com

9 http://www.sourceforge.net

10 http://trac.edgewall.org