Test Script language

<trace, expexc, actval, expval, type>

with the following interpretation:

1. trace:a sequence of functions or procedure calls.

2. expexc: the name of the exception that trace is expected to generate (or noexc if no exception is expected).

3. actval: an expression (typically a function call) to be evaluated after trace and whose value is taken to be the "actual value" of the trace.

4. expval: the value that actval is expected to have.

5. type: the data type of actval and expval.

Consider the stack package contained in stack.ads, and stack.adb. Below are two test cases for the stack package.

<s_init().g_top(), empty, dc, dc, dc>

<s_init().s_push(10), noexc, g_top(), 10, int>

A test plan and a complete test script for stack is shown in stack test plan and stack script. The module section defines the package prefix, which APE places in front of every subprogram. The accprogs and exceptions sections define the list of subprograms and the exceptions of the package. The globcod section contains global Ada code, delimited by the symbols {% and %}. APE places this global code at the top of the generated test driver. The test programmer can use the global code for with/use clauses, stubs, and utility functions that are called from the test cases. Finally, the cases section contains the test cases.

A test script may be viewed as a partial specification for a package, expressing its required behavior under specific circumstances. The purpose of APE is to generate a driver that will automatically determine whether a package implementation satisfies this partial specification.

Test program generation

The system flow for APE is shown below: ovals indicate human-readable files and boxes indicate executable programs.

The test programmer prepares the script using a text editor. APE reads that script and generates the ADA driver test.adb, which is compiled and linked with the package implementation. For example, for stack, the test script is stored in the file stack.script, and the package implementation in stack.adb. When test is executed, it runs the test cases from stack.script and reports any errors.

To generate the test driver, APE first generates code to record exception occurrences. Then, for each test case of the form

<C1...Cn, expexc, actval,type >

APE generates code that performs the steps outlined below.

invoke C1,...,Cn, monitoring exception occurences

compare the actual occurances to expexc

if there are any differences
        print a message
else
        if actval /= expval
                print a message
        if any exceptions have occured since Cn was invoked
                print a message

update summary statistics

Following the last case, code is generated to print summary statistics.