Model vs. GUI Testing in Smalltalk Applications

Author: Mark L. Murphy, The Sapphire Group (71202.2241@compuserve.com)

The first step to organizing a quality assurance program is to create a quality plan, where the project identifies what techniques it plans to use. One of the key decision points for any GUI development environment, including Smalltalk, is how much emphasis to place on testing the models (business objects) versus the views. I firmly believe that one should spend more time testing the model side, as GUI testing is inherently difficult and expensive.

There are many reasons why it is tough to implement a robust GUI test, including the following:

1. Automation: Many tools exist to perform GUI test automation, such as Microsoft Test for Windows. However, the genre is limited mostly to record-and-playback tools. These limit flexibility to the operations conveniently done from a GUI, rather than the operations the program is capable of doing.
2. Coverage: Perhaps the most powerful diagnostic aid in test assessment is coverage analysis. It is as easy to collect coverage statistics for a full GUI-based application as it is for a subsystem. It is much more difficult, however, to determine how to test missed areas, since coverage deals with coding constructs, not user actions. One must determine how some function, perhaps three layers deep in the application, can be triggered in a certain way by the user interface.
3. Setup: Tests on models tend to be smaller in scope than tests on GUIs. Setting up test data and environments for models, therefore, is frequently easier than for the entire application. This makes regression tests on GUIs more of a challenge, with a corresponding reduction in regression test frequency.
4. Stability: Tests for models and GUIs are expected to require changing when the underlying code changes significantly. GUI tests, however, may be less stable with respect to environmental changes. If the test result comparison is done by bitmaps of windows (images), for example, changes in system font, default colors, or monitor resolution may cause spurious test failures of otherwise correct code.
5. Portability: In part due to some of the above problems, porting tests on GUIs may be difficult. In the bitmap-comparison example, the changes in "look" between Windows and OS/2, for example, will cause the bitmaps to change.

This is not to say that GUI testing is useless, but that it should be employed solely for testing GUI features and less for application functionality. A view-level test, when testing application logic, should focus on "macro" operations, such as use cases, rather than "micro" features, such as whether a particular record is stored in the database properly.

There are two big arguments in favor of heavy GUI-level testing, even to the point of skipping model testing altogether:

• It saves the overhead of doing another test plan and implementation. While this may be true, the effectiveness of GUI-level testing is comparatively low. Hence, to achieve the same overall level of quality, one might spend more on an all-GUI-test quality program than one based on model testing.
• GUI testing, courtesy of capture-and-playback tools, typically requires little to no programming and can be conducted by people without Smalltalk skills. This is a valid argument, particularly with solid Smalltalk developers in fairly short supply. However, one must be careful not to simply use this as an excuse for sub-standard quality assurance.

Over time, the increased use of frameworks and other reused components will reduce the amount of new code any project creates, and hence eases the testing burden. Until some magical time that testing becomes obsolete, however, one should focus on testing the models first and foremost, to get the greatest quality benefit from the least effort.