Evaluating Emerging Software Development Technologies: Lessons Learned from Assessing Aspect-oriented Programming
Gail C. Murphy, Robert J. Walker, and Elisa L.A. Baniassad

IEEE Transactions on Software Engineering, 25(4):438--455, July/August 1999. Special Section on Empirical Software Engineering.

Abstract

Determining whether a new software development technique is useful and usable is a challenging task. Various flavours of empirical study may be used to help with this task, including surveys, case studies, and experiments. Little guidance is available within the software engineering community to help choose among these alternatives when assessing a new and evolving software development technique within some cost bounds. We faced this challenge when assessing a new programming technique called aspect-oriented programming. To assess the technique, we chose to apply both a case study approach and a series of four experiments because we wanted to understand and characterize the kinds of information that each approach might provide. In this paper, we describe and critique the evaluation methods we employed, and discuss the lessons we have learned. These lessons are applicable to researchers attempting to assess new programming techniques that are in an early stage of development.

Superceded Versions

Gail C. Murphy, Robert J. Walker, and Elisa L. A. Baniassad. "Evaluating Emerging Software Development Technologies: Lessons Learned from Assessing Aspect-Oriented Programming", Technical Report TR-98-10. 34 pages.
[PDF], [gzipped PostScript]

IEEE Copyright Notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.