Participating 
     Organizations 
  Research   
     Topics 
  People 
   

formalWARE 
    products  

  Publications 
  Presentations 
  Tools   
  Methods 
  Examples   
  Training 

formalWARE  
  information  

  News 
  Events 
  Index  
  Search   
  FAQ   
  Links   
  Contacts

Introduction 

 The tests for package P must be repeated many times; for the initial version of P, after each modification to P, and when P is used in a new environment, including a new operating system or a compiler. This requirement for repeatability suggests that test execution should be automated.  

It is typically difficult to thoroughly test a package P while it is installed in a production system. P's subprograms are often not directly accessible. If P is a general-purpose package, some of its subprograms may not be called at all in a particular production system. Errors in other packages may appear to be errors in P. Conversely, errors in P may be masked by errors in other packages. Using test scaffolding, P may be tested in isolation from the production environment. In practice, packages are best tested with a mixture of scaffolding and production code; the critical tradeoff is between the benefits realized through isolation, and the cost of developing and maintaining the scaffolding. 

Although implementing test drivers manually is straightforward, it is also time-consuming, repetitive, and error-prone, and it produces code that is costly to maintain. As a result, test driver generation is a good candidate for automated support. For most packages with a call-based interface, we use the testing tool APE to generate batch test drivers from test scripts. 

"APE" (Ada Package Exerciser) automates some aspects of developing Ada code into into unit-testing packages. Following the specification of test cases in a more concise format than is normally done when writing Ada code directly, APE automatically expands into Ada code a test script that can be compiled and linked with the package being tested. 

Preliminary results, following the experimental application of the tool to some CAATS software components in the APEX development environment indicate that the APE input scripts are much more concise than Ada code written to unit-test a software component, while achieving the same level of test coverage. As well as the potential for significant reduction in the amount of effort required to conduct unit testing, this tool is likely to yield greater consistency in the way unit tests are written. 

Resources 

APE Tutorial 

This tutorial gives an introduction to the APE scripting language, an example using a stack package and instructions on how to use APE for testing. 

Automated Package Testing 

A presentation given by Jayakrishnan Nair at the formalWARE Open House, UBC on Oct. 8, 1997. 

Recommended Reading List 

• D. Hoffman and P. Strooper, "A Framework for Automated Class Testing" , Software - Practice and Experience, Vol. 1(1),1-4(December 1996) 
• G. Murphy, P. Townsend and P. S. Wong, "Experiences with cluster and class testing", Communications of the ACM,37(9),39-47(1994) 
• D. M. Hoffman and P. A. Strooper, "The testgraphs methodology - automated testing of collection classes", Journal of Object-Oriented Programming, 8(7), 35-41(1995) 
• D. J. Panzl, "A language for specifying software tests", Proc. AFIPS Natl. Comp. Conf. AFIPS, 1978, pp 609-619. 
• J. Gannon, P. McMullin and R. Hamlet, "Data-abstraction, implementation, specification and testing", ACM Transactions on Programming Languages and Systems 3(3), 211-223(1981) 
• D. M. Hoffman, "A CASE study in module testing", Proc. Conf. Software Maintenance, IEEE Computer Society, October 1989, pp 100-105. 
• D. M. Hoffman and P. A. Strooper, "Software Design, Automated Testing and Maintenance: A Practical Approach", International Thomson Computer Press, 1995 
• D. M. Hoffman and P. A. Strooper "Automated module testing in Prolog", IEEE Transactions on Software Engineering, 17(9), 933-942(1991) 
• P. G. Frankl and R. K. Doong, "The ASTOOT approach to testing object-oriented programs", ACM Trans. on Software Engineering Methodology, 3(2), 101-130(1994) 
• S. P. Fielder, "Object-oriented unit testing", Hewlett-Packard Journal, 69-74(1989) 
• M. J. Harrold, J. D. McGregor and K. J. Fitzpatrick, "Incremental testing of object-oriented class structures", Proc. 14th Int. Conf. on Software Engineering, 1992, pp 68-80. 
• R. V. Binder and W. A. Fuson, "Testing, in a perfect world", Communications of the ACM, 37(9), 78-86(1994) 
• B. Marick, "The Craft of Software Testing", Prentice Hall, Englewood Cliffs, NJ, 1994  

  ---

Recommended Web Sites

Standard for Software Testing 

The objective of this Standard is to enable the measurement and comparison of testing performed on software components. This will enable users of this Standard to directly improve the quality of their software testing, and improve the quality of their software products. 

Object Oriented Testing 

A list of print and electronic resources on this subject 

Testing Strategies and Methods 

A discussion of resources for conventional software testing 

Software Testing Home Page 

The Software Quality Engineering and Test (SQET) Team offers products and services to adopt effective software testing and test-related technologies. Also available on this Web Page are documents and articles related to testing. 

Software Test Management 

This document at the Software Engineering Institute at Carnige Mellon University describes the method to plan and monitor the software testing effort and control the testing resources. 

Software Testing Institute 

Quality publications, industry research and online services for the software development and testing professional. 

Software Testing Hotlist 

Resources for professional software testers.  Bret Pettichord, editor 

STORM - Software Testing Online Resources 

A nexus of Software Testing Online Resources, this site is designed to be a "first-stop" on the Web for software testing researchers and practitioners. 

 

This page has been accessed times since November 7, 1997.