Code-Level Testing of Smalltalk Applications

Author: Charles Weir, Object Designers Ltd. (cweir@cix.compulink.co.uk)

Copyright (c) Charles Weir 1995

INTRODUCTION

This paper discusses some of the issues associated with testing component parts of Smalltalk applications. It examines means of making code test itself, ways to direct testing for maximum effect, and describes some of the issues associated with managing and reproducing test procedures.

WHY DO CODE-LEVEL TESTING?

The purpose of testing is to find bugs. Smalltalk has a background as a developers' language; code debugging is outstandingly easy. This has led in many projects to a gung-ho attitude to development: we design the system, roughly specify the components, implement them, and test the result using the user interface with maybe simulations for other interfaces. In short, we tend to stick to system-level testing. This approach works well, up to a point. Good tools are available to simulate user input; also many Smalltalk applications have been predominantly user interface processing, so this tests much of the code effectively.

However, in a large system, and particularly one where the underlying functionality is complex, it becomes very difficult to test everything through external interfaces. A more cost-effective approach is to segregate internal components and to test using software interfaces - to test using the code itself.

CODE SELF-TESTING: IMPLEMENTING ASSERTIONS

A simple technique is to make the code do its own self-testing, using assertions. This is a code statement that is always true; it tests that the code is behaving as expected. If the statement is false, the assertion stops processing to allow debugging.

An implementation of assertions is Smalltalk/V is: !Context methods ! assert self value ifFalse: [ self halt ].! !

This allows us to verify assertions wherever required, for example:

     [ range first == 1 ] assert. 

Of course, in the release version of the system, these tests are redundant and waste processing time. So we replace the method Context>>assert with a null version:

     !Context methods !
     assert
         ^ self! ! 

The overhead of the extra method dispatch is relatively small; of course, if it is still significant, we can write a utility to find the senders of method #assert and to comment out the assertion statements.

Assertions are particularly useful for testing preconditions, postconditions, and class invariants. See [Meyer] for a discussion of these ideas, and [Cook&Daniels] for a design method which uses these extensively.

COMPONENT TESTING

While self-validation helps with problems which may not be found quickly during system testing, it does not help at all for the parts of the code which system testing does not reach. For this, we must produce separate code explicitly to exercise the functionality. This is code-level testing.

SELECTING WHICH COMPONENTS TO TEST

Given the ease of debugging Smalltalk code, and of executing statements on the fly, indiscriminate code-level testing is a waste of time. To optimise the effect of testing, we need to be selective. Specifically (see [Love]), we test code that is:

1. Complex, and
2. Insufficiently exercised during system testing.

To identify unexercised code, we need source coverage analysis; these are generally available, if only as performance monitoring tools.

REQUIRED: COMPLEXITY MEASUREMENT FOR SMALLTALK CODE.

Currently, I am unaware of any automated complexity metrics for Smalltalk code. In default, we can use a tool such as an 'awk' script to count average numbers of non-comment lines per method, and numbers of methods per class. The components scoring highly on both are those likely to justify code-level testing.

IMPLEMENTING SOURCE LEVEL TESTS

To be cost-effective, source level tests must require minimum maintenance and effort from future developers. This implies that:

• They must be self-contained
• They should do their own validation (answer "Pass" or "Fail")
• We need a means to execute all the code tests for the application in one suite.

The main substance of each test will be written as a single method. The tests may do their own validation using assert statements; even text output can be checked using file comparison:

     [ OperatingSystem
         executeCommand: 'compare file1 file2' ] assert. 

A common containment idiom is to implement the tests for one or more classes as a class method on one. However a more satisfactory approach is to define a specialised test class for each test. [Beck-1994] describes a simple framework for this, where we create tests by subclassing the class TestCase, and combine them in instances of TestSuite.

Either approach then gives an effective and reproducible test suite for the system.

REFERENCES