Automated generation of test programs from closed specifications of classes and test cases

Abstract

Most research on automated specification-based software testing has focused on the automated generation of test cases. Before a software system can be tested, it must be set up according to the input requirements of the test cases. This setup process is usually performed manually, especially when testing complex data structures and databases. After the system is properly set up, a test execution tool runs the system according to the test cases and pre-recorded test scripts to obtain the outputs, which are evaluated by a test evaluation tool. This paper complements the current research on automated specification-based testing by proposing a scheme that combines the setup process, test execution, and test validation into a single test program for testing the behavior of object-oriented classes. The test program can be generated automatically given the the desired test cases and closed specifications of the classes. With closed specifications, every class method is defined in terms of other methods which are, in turn, defined in their own class specifications. The core of the test program generator is a partial-order planner which plans the sequence of instructions required in the test program. The planner is, in turn, implemented as a tree-search algorithm. It makes function calls to the Omega Calculator library, which solves the constraints given in the test cases. A first-cut implementation of the planner has been completed, which is able to handle simple arithmetics and existential quantifications in the class specifications. A soundness and completeness proof sketch of the planner is also provided in this paper.