Teaching Software Engineering through Real Projects
Desanka Polajnar and Jernej Polajnar
Department of Computer Science
University of Northern British Columbia
Prince George, BC, V2N 4Z9
{desa,
polajnar}@unbc.ca
ABSTRACT
Software engineering instruction often relies on semester-long student team projects in trying to recreate a realistic experience of professional software development in a classroom setting. A common weakness arises in the emulation of customer role within the requirements engineering process. This paper explores the merits and risks of teaching requirements elicitation skills through term projects with real external customers in the case of smaller class sizes. The analysis draws from concrete experiences of the authors. In the framework of a two-course software engineering sequence in the third year of an undergraduate computer science curriculum, the focus is on a requirements engineering project within the first course. The project organization is described, followed by a discussion of different approaches to the emulation of customer role. The approach involving real customers is illustrated with a comparative case study of two student teams working on parallel projects within the same course and with the same real customer. Some general conclusions are offered on how one might achieve a balance between exposing the students to a realistic requirements elicitation experience and keeping the project load, pace, and profile within the boundaries of the course syllabus.
Keywords: Software engineering education, requirements engineering, student term projects, requirements elicitation, customer role.
1. Introduction
Software engineering courses often include team projects,
in which instructors try to recreate, on a small scale
and in classroom environment, a realistic sense of the tasks
and phenomena that naturally arise in larger, real-life projects.
At the same time, they try to preserve, as much as possible,
the controlled classroom setting with carefully paced progress,
predictable completion times, adherence to the course syllabus,
and reasonable grading curves.
These frequently conflicting requirements make software
engineering
courses hard to teach.
In trying to emulate professional software development in the
classroom, one of the weakest spots is requirements elicitation.
Whether the sources of domain knowledge and customer expectations
happen to be the team members themselves,
other students,
members of the teaching staff, invited guests,
or external interviewees, the `virtual'
customer is usually a poor substitute for the real one.
While virtual customers may
or may not have genuine expertise in the application domain,
their requirements typically
bypass many areas that are relevant and difficult to address
in real projects.
Such areas include legacy system
or personnel constraints,
preferences for preserving existing operating procedures,
itemized scrutiny of individual features based on cost,
sequencing imposed
by budget schedules and, in particular, semantic gaps
between customers and developers that in reality
often pose significant peril.
The lack of a realistic requirements process
is a serious problem, since the elicitation of project
requirements is one of the
more critical skills that software engineering curricula
are supposed to teach.
In this paper we explore one possible approach to the problem,
namely the adoption of class projects that are done for real
external customers (cf. [1]).
Those customers may or may not belong to
the educational institution offering the course, but their
own objectives must be real and unrelated to the
pedagogical objectives of the course.
They must stand to seriously benefit
from the product and have a genuine interest in it.
When pondering how to fill the customer role in
a class term project, one needs to find a balance
between, on one side, the desire to expose the students
to a realistic experience
that helps develop their requirements elicitation
skills -- and, on the other side, the need to keep the project's
workload, pace, profile, and outcomes within
the tight limits imposed by the one-semester course syllabus.
In this paper we examine some of the factors that
affect this balance, in the context of our teaching experiences
based on smaller class sizes.
We describe the organization of a requirements engineering
term project used in a specific software engineering course,
analyze different ways of filling the customer role in the
project's requirements elicitation process, follow the
progress of two teams working on parallel projects in the
same course with the same real customer, and use those
concrete experiences to suggest some general conclusions.
2. Term project organization
Our analysis is based on existing experiences with
smaller software engineering classes consisting of computer
science majors at the upper-division undergraduate level.
They include our observations from teaching CPSC 300/301
Software Engineering I and II (first author), as well as
research (CPSC 495/496) and project (CPSC 490/491) courses
with software engineering topics (second author) at the University of
Northern British Columbia. In this paper we shall mainly refer
to the Software Engineering two-course sequence. The main
subject of the present section is the term project
organization in the first course.
In this section we analyze the ways
in which the customer role can be filled for the
purpose of requirements elicitation in the class project.
In particular, we focus on the distinction between
`real' and `virtual' customers. A more complete
classification of customer roles, with analysis and characteristic examples,
is presented in [4].
In this section we discuss specific experiences with term
projects in which the requirements elicitation process involved
real customers. We study the case of two student teams in Software
Engineering I, working on parallel projects with the same real
customer during the same term. The projects followed the
organization described in Section 2.
The customer was a University
research team in environmental science performing large
simulation experiments in the area of climate studies.
The customer was cooperative and prompt in providing information.
The environmental research team did not include computer scientists, and the
software engineering student teams had no prior knowledge of the application
domain, resulting in an initial semantic gap.
In this paper we have explored the benefits and risks of
relying on term projects with real customers in software
engineering courses. The background for our analysis includes
some of our own observations from teaching software engineering
topics to undergraduate computer science majors. In particular,
we discuss the ways of emulating the customer role in
a requirements engineering term project, and report experiences
from projects with real customers.
Software Engineering I and II are electives that are strongly
recommended to Computer Science majors. Typical enrollment is
15-20 for the first course, somewhat less for the second. The
student population is a mixture of third and fourth year students.
All have completed the four lower-division courses on programming,
algorithms, and data structures, a course on system analysis and
design, and (usually) a course on user interface design. Some
have also had upper-division courses on operating systems,
database systems, programming languages, and other software areas.
They are advised to get a balanced background before entering
the software engineering courses.
The first course is self-contained (but
designed with the expectation that most students will
continue into the second course).
It has a substantial theoretical
component (based on [2, 3]).
Its emphasis is on software process models,
system models, and requirements engineering.
The project component focuses on the requirements
engineering phase of software development.
It includes requirements elicitation,
generation of requirements
documents that comply with the IEEE/ANSI 830-1993
standard [2], development of a limited
prototype (not `throw-away'),
definition of test cases for a key subset of the product's
requirements specification, and validation of requirements.
The second course
includes detailed design of selected functionality
(based on UML models),
implementation, and testing. Design and implementation
follow an incremental
process that started with the prototype in the previous term.
The student project teams typically consist of three to five
students each. They are formed in the second week of
classes by students themselves, with occasional balancing
intervention from the instructor.
The project topics are determined in the third week of classes.
An initial list of choices is proposed by the students and the
instructor, and discussed in class. Each team then presents
a prioritized list of three topics to the instructor. After
the resolution of conflicting choices and some further
discussion, each team adopts one topic. When approving the
topic, the instructor assists the team in appropriately
scaling the project size. By that time, the lectures
have covered an overview of the requirements engineering
process.
In its initial meeting, each team is required to do the
following:
The team leader negotiates the frequency of meetings
with the instructor and with the customer.
In the first joint meeting with the course instructor
and the customer, the instructor's role is to demonstrate
to the team how to interview the customer.
Within two
weeks from the topic approval, students may request to
switch teams in consultation with team leaders
and the instructor.
Project topics proposed by students are sometimes
planned well before they enroll in the course.
They reflect individual or team interests and in
some cases explore project ideas that are later
applied in professional context. In this paper we
are primarily interested in situations where
there is a real external customer interacting with
the student team during the requirements elicitation stage.
3. Requirements elicitation with virtual and real customers
A virtual customer is someone who is willing to
play the customer role as a part of the requirements
process in the course, but is not a real stakeholder or
user of the software product. This can be the course
instructor, teaching assistant, a student not belonging
to the team, a fellow faculty member, or other volunteer.
Requirements elicitation with virtual customers involves
several potential difficulties. One is a possible lack of
depth in the application domain knowledge. This need not
be the case. For instance, the course instructor may have
relevant professional experience in the application area,
or the team may be lucky to recruit a domain specialist
as volunteer. Another danger is that the virtual customer
may know too much about software development to be able
to model the semantic gap that typically arises in real
life. Moreover, not having a stake in any particular
features of the product, such a `customer' might in fact
be guided, partly at least, by process rather than
application needs. An external domain specialist may be
sufficiently removed from software development to provide
a semblance of a real semantic gap, but will likely act as
a disinterested consultant friendly to the team rather than
as a firm negotiator defending own perceived interest.
In addition, the specialist consultant may not be available
for sufficiently frequent regular meetings.
An inherent downside of virtual customers, even assuming
independence and domain expertise, is that they primarily
emphasize the general functional requirements. Realistic
performance requirements are often difficult to emulate.
Safety concerns, workplace regulations, training and
practices of existing personnel, administrative structure of
the client organization, legacy equipment and software,
externally dictated timelines -- these are some of the
constraints typically present in real-life requirements
processes that cannot be realistically expected of an
elicitation process involving virtual customers.
A real customer is a stakeholder
who is expected to use the product,
stands to benefit from it, has a direct,
genuine interest in the product requirements,
and wants a decisive control over them.
The requirements of a real customer do not capture only
the set of intended functional `features' of the product,
but also insist on its practical usefulness within a realistic
set of constraints.
The real customer normally has the requisite
domain knowledge, but may speak a rather different
language than the project team, and may have
insufficient knowledge of the technology to optimally
set the level of specific requirements.
Misunderstandings with a real
customer can cause real harm. This provides the
necessary subjective sense of risk, but may also introduce
risks to the pedagogical outcome of the course.
A further realistic feature which is seldom present
in courses is the existence of a formal contract
between the customer and the team. If the team is
bound by contract to produce deliverables that meet
the requirements specification, their experience is
closer to reality but the risk is also enhanced.
Requirements
engineering with virtual customers has inevitable
shortcomings, but allows the course instructor to do
the necessary planning and maintain control.
Requirements engineering with real customers is clearly
preferable if the risk of the course getting out of control
can be kept to a minimum at a reasonable cost. In the
next section we explore that possibility through specific
project examples.
4. A case study involving real customers
Before the topic definition stage, the customer presented
an overview of their research to the entire class and the
course instructor. The talk also introduced the two simulation
systems and the visualization system they use, and identified some
desired improvements. In particular, the two simulation systems
generated their output files in different formats, only one of
which was accepted as input by the visualization system. This
limited the possibilities of visual comparison of the simulation
results produced by the two systems. Furthermore, the customer
controlled the visualization system through a command-line interface,
relying on memorized syntax of various weakly
documented commands and their parameter lists. The overall
impression was that the power of the simulation and visualization tools
was exploited to a limited extent due to weaknesses in interfacing.
In the topic definition stage, the course instructor explained
the likely advantages and disadvantages of working with real customers
in a term project. The advantages included: a more realistic
elicitation process, resulting in valuable requirements engineering
experience; learning how to acquire domain knowledge from a specialist;
and the domain knowledge itself, including the approaches, tools,
and methods used in climate research. The disadvantages included:
more time invested; more revision of initial requirements as the
context becomes better understood; and risks of misunderstanding the
customer and the problem, which could compromise the project objectives.
In order to alleviate the risk, the instructor offered to attend
several early meetings with the customer (taking the `back seat'
and helping ensure that all key issues are adequately covered).
The instructor did not make any suggestions to the teams as to
the topics they should select.
In the topic selection stage, two three-member
teams expressed interest in working on some of the presented
problems with the real customer. Team 1 was to address the issue
of format conversion and better connect the simulation and
visualization stages. Team 2 was to determine the full capabilities
of the visualization system and build a graphical user interface (GUI)
for it. Each team was to have several meetings with the customer
until the task and the context were fully understood, draft the user
requirements and system requirements documents, have them validated
by the customer, and then proceed to build a limited-functionality
prototype that should be validated by the customer and used as a
basis for detailed design and implementation in the follow-up
Software Engineering II in the second term.
As they set to explore the background, the students encountered
various difficulties typical of real projects. The systems were
public-domain, open-source free software, developed over several
decades in a decentralized manner by a number of researchers in
the application field, and still evolving. The programming language
choices (Fortran being gradually replaced by C), the lack of
architectural uniformity, and the lack of systematic documentation
were characteristic of the systems' life cycles. As a result, it
was difficult for a system user to get a complete insight into the
system's capabilities, and it seemed that a typical user would
probably be familiar with only a subset of its full functionality.
At the end of the term, both teams successfully met their
requirements engineering objectives. They both proceeded with
successful detailed design and implementation stages in the second
term and came up with useful deliverables that met (and in some
aspects exceeded) the initial expectations. However, their paths
in getting to the goal were rather different.
Team 1 took a cautious approach and invested significant time
into early interactions with the customer. The course
instructor attended the meetings and initially ensured that
the interview process covered all the necessary angles. The
instructor was subsequently consulted when the parties had
doubts about mutual understanding of details, but the need for
such intervention gradually disappeared. The initial requirements
documents that the team generated were approved by the customer.
The team collected information about the relevant file formats
and inspected the legacy code to extract the data formatting
detail, clarifying the system architecture in the process.
They went on to design and implement a limited working prototype
of the converter utility in C++, with its own graphical user
interface in Java. The prototyping phase revealed further
imperfections in the requirements documents, which were
subsequently polished to provide a solid basis for the design
and implementation phases.
Team 2 started with a more confident approach, possibly because
their task seemed less intricate at the outset. Following the initial
meeting with the customer and the course instructor, they started to
work on the requirements documents. They had just one additional ad hoc
meeting with the customer alone, and reported to the instructor
that everything was clear. Their interactions were not thorough enough
to let them appreciate the semantic gap, and they were shocked to find
their draft requirements documents rejected by the customer. They
believed in their proposed solution and only belatedly understood that
that was not what the customer had been asking. The elicitation process
had to be rolled back. Fortunately, the team managed to recover,
revise the requirements documents, and get back on schedule. It should
be pointed out that the rescue was accomplished without a need for
instructor's intervention. The team inspected the legacy system code
to extract the full command-line syntax and identify its full
functionality. They went on to implement the prototype,
and later a more complete version, of the GUI in Java, using the Java Native
Interface to connect with a system routine in C that invoked Unix kernel
calls to interact, through standard input/output, with the command-line
driven visualization system.
One can conclude that in this case the student teams were exposed to a
rather realistic experience in the requirements elicitation process,
while at the same time the project work did not disrupt the balance
within the course syllabus. As far as we know, the performance of
team members in their other courses was not affected. The shortcut
that one of the teams took may have coincided with time pressures in another
course. However, on the whole the project activity seemed to fit well
into the course and the students' study load. The students reacted with
satisfaction, in particular at the end of the second course as they completed
the implementations.
How should this positive result be interpreted and does it suggest some
general conclusions? One should first note that the course instructor
was able to closely monitor each team's progress and interact with teams
and customers in a timely manner. With a total of four teams, this was a
substantial task. In a larger class one would need competent teaching
help, e.g., graduate teaching assistants specializing in software
engineering, to maintain comparable attention to project dynamics.
An important aspect of such attention is that each project was initially
carefully scaled to an appropriate size and level of difficulty;
this is what allowed a team to recover from early requirements errors.
In order to help properly scale the projects, the instructor needed
a good insight into the students' preparation and capabilities.
The customer in this case study was real in the sense of having
a strong interest and commitment to the project, planning to directly
use the product, being
independent of the course, having the application domain knowledge,
and being available for detailed interaction. However, several
factors moderated the possible project pressures: the customer did not
directly fund the work, understood the academic limitations and agreed
that they should be respected, and did not have their own progress
critically tied to the project outcomes. In our view, this was a
reasonable balance that provided a good sense of reality while
keeping the risks manageable.
5. Conclusions
The analysis emphasizes the importance of finding a proper balance
between exposing students to realistic experiences and ensuring that
the project's workload, pace, and profile remain within the
reasonable boundaries as outlined by the course syllabus.
Some of the relevant factors are: the class size and the ability
of instructors to closely monitor individual projects; careful
initial scaling of team projects, based on sufficient insight
into the students' educational backgrounds, practical experience,
and potential; choice of real external customers who are able
and willing to respect certain limitations of course projects,
are not under extreme time pressures, have a strong interest
in the product, and expect to be available for timely interactions.
With the proper stipulations, precautions, and resources in place,
the value of having real customers in requirements engineering
class projects can hardly be overestimated. A realistic software
project experience is not only a necessary prerequisite for
grasping the intent of software engineering methods, but also a
prerequisite for proper understanding of why the discipline exists.
One learns most from unexpected situations, and project organization
should allow for them to happen. A study of software engineering
methods in an overly controlled environment may create a false
feeling of security -- a rather undesirable educational outcome.
Working with real customers is definitely one way of maintaining
a healthy balance between theory and experience.
References
Name Desanka Polajnar Jernej Polajnar
Department Computer Science Computer Science
Institution University of Northern British Columbia
University of Northern British Columbia
Postal address
3333 University Way 3333 University Way
Prince George, B. C. Prince George, B. C.
Canada, V2N 4Z9 Canada, V2N 4Z9
E-mail address desa@unbc.ca polajnar@unbc.ca