1. Introduction

In this paper, I discuss my experience using WebCT [GSS96] as a supporting tool for delivering a graduate course in computing science in parallel to the campuses of the University of Alberta and the University of Calgary. This parallel delivery was part of the activities planned in the context of ALSE (Alberta Learning Node for Software Engineering) a LEE3 project between the two Universities.

The  ALSE project investigates the activities involved in developing and delivering electronic courseware, with emphasis on

1. the identification and qualification of learning material by domain experts,
2. the construction of particular courseware for an interactive distributed delivery system by content providers, and
3. the guided/monitored access to reference material and interactive services by students.

As part of the evaluation of our work in ALSE, we have identified three different dimensions that we expect to play important roles in effective electronic delivery:

1. the level and the pedagogical objectives of the course, i.e., whether it is undergraduate or graduate and whether it is mainly text-driven or project-based
2. whether or not the electronic material is used in parallel with classic classroom delivery, and
3. the actual tool supporting the delivery of the electronic material.

Given this hypothesis, we have designed an evaluation plan for exploring this three-dimensional space. We have selected a set of four courses -both at the undergraduate and the graduate level- across the two campuses. All the four courses in this initial set, are in the area of Software Engineering, because this is the primary research area of the project team and in order to avoid affects related to the nature of the subject matter and the degree to which it may be amenable to using electronic material. Some of these courses are being delivered within a single campus in a regular classroom format, where others are being offered in "smart classrooms" in order to enable parallel remote delivery from one campus to the other. Finally, for all of these courses, we are developing electronic material  in two different formats to be used in combination with two different electronic courseware tools, WebCT and WebStract.

2. The Original Course

The course on which I report in this paper, UoA-CMPUT 661, is a graduate level course, on Software Architecture. In addition, a version of this course is also being part of the WestMOST program. The subject area that the course deals with is fairly new, and there is a lot of active research currently in progress in academic and industrial institutions all over the world. As a result, relevant electronic material is being made available every day on the WWW, which makes the course a quite appropriate test bed for ALSE.

The objectives of the course are

1. to familiarize the students with the concept of architecture styles, their properties and the types of problems for which they are most appropriate,
2. to examine different formalisms for specifying a software architecture, and
3. to examine the role of software architecture throughout the software development life cycle.

The coursework involves studying and presenting technical papers, aimed towards accomplishing the first two learning goals, and working on a team project, intended to accomplish the third learning goal. Thus it enables the investigation of the affects of integrating electronic delivery tools across the spectrum of text-based vs. project-driven learning.

3. Course Design

During winter term 1999, this course is being offered in a smart classroom at the University of Alberta and, at the same time,  it is synchronously delivered to the University of Calgary, through a compressed videoconferencing. It runs for three hours, once a week, and there are 17 students registered at the UofA and 7 at the UoC.  WebCT is used to enable asynchronous interaction in   parallel to the class meetings. This particular offering presented a set of special challenges. The first was the length of each individual class, along with the fact that it was delivered in the evening, after a whole day's work for all the participants. This format demanded the development of special strategies in order to keep the interest of the participants throughout the class. At the same time, an important requirement was to involve the students in the two campuses in, as much as possible, the same way, so that neither group was at any disadvantage. Finally, a third important requirement was to naturally integrate the use of electronic materials in support of the course learning objectives. A large body of electronic material had already been developed for the previous offerings of this course, including HTML and PowerPoint presentations as well as an extensive list of references and pointers to relevant WEB sites. This material had of course to be revisited and redesigned within the WebCT environment.

My approach to these challenges was to develop a general class plan that intersperses lecturing with student presentations and free discussion. For every weekly meeting a story board was developed, listing the main issues to be discussed and the time to be devoted to each one of them. Two student teams, of 2-4 people, are actively involved in each meeting: one team presents a technical paper and the other addresses a "controversial" issue and posits specific questions for the general audience to discuss. The paper presentation is intended to be in-depth and to precisely represent the position of the authors within 25 minutes. The underlying goal is to provide technical depth on issues related to the subject of the week. On the other hand, the debate presentation is meant to be short -approximate 10-15 minutes- and broad and to introduce new as many relevant points of view as possible, starting from the presenters' own. The pedagogical goal of the debate is to motivate the debaters to think originally and to allow the change the pace of the meeting by allowing the whole class to actively participate in an open-ended discussion. Both teams are given an initial set of references and the debate team is also required to locate more references relevant to the issue they are debating. The story board for each meeting is reflected in the weekly content entry of WebCT, and all the presentations -lecture notes, paper presentation and debate presentation- are made available on line a few hours before the actual meeting.

4. The Instructor's Experience

This section discusses a few aspects of the course as experienced by the instructor. In particular, it identifies a set of areas in the development of the new course format that required substantial effort.

The first task was to set up the WebCT environment in a way that would best support the above course design. The design of the environment was kept simple, using a few WebCT tools organized in two layers. The tools used at the first layer were the course content presentation, the bulletin board, the calendar and the email. In addition, three more pages were added, indexing the paper presentations, the debates and the projects. The content of each week consists of a description of the weekly themes and story board, pointers to earlier versions of electronic course material which was made available in the beginning of the course, and description of the paper and debate presentations. The rationale for this design was to enable quick access to course materials in order to foster on-line continuations of the discussions started in class. More WebCT tools were made available through an "additional tools" link on this front page.

The second area requiring major effort was the redevelopment of the course materials. During previous offerings, I had taken an incremental approach to distributing materials, mainly in order to allow for introduction of new material as it was being discovered. However, in this format a lot more planning was required in order to avoid delays in the delivery of paper-only materials to the remote site. So, I revisited the reading list as it had evolved through the last course offering, identified these items that were available only in paper and sent them to the remote site in the beginning of the course. No more paper materials are being introduced during the course, but electronically available materials are continuously being added.

Having set up the infrastructure for the course, the weekly tasks for maintaining WebCT involved preparing the weekly story board, preparing for the lecture, posting and indexing the notes for the lectures and the presentations, and monitoring the on-line discussion. A rather conservative estimate of the time needed for these tasks is approximately ten hours per week.

5. Students' Experience

In order to understand the course audience better and also to monitor the progress of the course three types of information are collected from our students: a personal profile, a weekly study log, and an exit summative evaluation survey.

At this point, since the course is still in progress, only the profile information and part of the weekly logs is available. From the student profiles it can be seen that there is a lot of diversity among the course students. The UoC students are working in parallel to their graduate studies where all of the UoA participants are full-time students. Their reasons for taking the course were partly interest in the subject, partly degree requirements fulfillment with most of them stating that it was an interesting available option. Some of the students have had previous experience with WebCT but most of them did not. The experience of the students in terms of system development covered a wide range from course projects to legacy system reengineering. Finally their professional goals ranged from research to software design and development to consulting and running their own business.

The weekly log was designed to provide a better understanding of the affects of the course and the tool environment design. The students were asked to fill out a weekly log with their impressions of the week. The log form consists of three very general questions, on the scientific outcomes of the week, how much time they spent working on the course, and in general, what worked and what did not work for any given work. Annotations on the form explained to the students that the purpose of the form was to collect their thoughts on the course content and organization, the quality of the chosen readings and presentations, and the quality of the delivery media. The logs were submitted to an ALSE collaborator who striped them of the identifying information and forwarded them to the instructor. From the logs submitted to date, the time students spend on the course varies between 0 to 16 hours per week, depending on their other workload, on whether they are presenting or not this week and on the phase that their project is in. Some students feel that this is a comparatively heavy workload but the comments to this effect have been relatively few. Most of the students are intrigued by the physical smart classroom setup and they make quite often comments about the quality of the video and audio streams, and also the instructor's handling of the technology. Especially in some instances when we had technical problems, the students registered in their logs quite strong complaints about the difficulty that these problems introduced in concentrating in class. Finally, the logs contain a lot of comments on the effectiveness of the readings and the in-class discussions, which will be valuable in the review of the content at the end of the course.

6. Preliminary Findings

As the course is still in progress, an overall evaluation is premature but based on the up to date experience to date I would like to offer a few preliminary findings:

1. Electronic media are expected to facilitate the participation of students to course-related discussions who may feel uncomfortable participating in class. I was especially interested to see this effect, but in this course, there has been no evidence of the phenomenon. The students who participate in in-class discussions are the same who continue the discussion off-line. There is one exception. a new student from China feels still uncomfortable with his oral skills but can express himself in writing.
2. From spurious discussions I had on occasion with local students, many of them find WebCT an extra hoop they have to go through in order to accomplish tasks that could have been accomplished with simpler tools like Web browsing and email. Because of the nature of the course, some of the more interesting tools that WebCT offers, like on-line quizzes are irrelevant. Furthermore, an ideal graduate course should generate discussion, and still most of the people find face-to-face discussions more compelling. Perhaps, with more exposure to this faceless interaction this predisposition will weaken.
3. The depth of the on-line discussions depends on the issue as well as the interest of the originator of the discussion. The presenting teams are supposed to kick off a forum with their topic at the end of the class. Few of the teams have met this mandate, and I have not been able to find a message to effectively motivate them. In spite of the fact that 10% of their final mark depends on their participation, a fact stated in the course outline, I have consciously decided to avoid mentioning the grading scheme in my invitations to contribute to the discussion.
4. As it has already been reported in the literature [Jacobsen98,Kristapiazzi1998], a lot of effort is required on the part of the instructor to manage the WebCT environment. In particular because of the parallel local and remote delivery the project monitoring is done through WebCT, and that process incurs a particularly high cost. The advantage is that all discussions are logged which makes for more precise exchanges but on the other hand this can be difficult especially in a course focused on design which is inherently open-ended. A technical issue that adds to the effort required for a course is the diversity of platforms that people use to develop their electronic materials. Use of proprietary-format tools such as PowerPoint means that a translation overhead has to be added in order to make the materials available across platforms.
5. Finally, I have found the task of managing the smart classroom technology quite daunting while at the same time lecturing or monitoring and participating in the discussion. The interface is unintuitive and the response is often too slow or too abrupt and requires compensation. In addition, such unexpected behavior distracts the students' attention.

Acknowledgments

The author would like to thank the ALSE team members, Paul Sorenson, Mildred Shaw, Werner Joerg, Rob Kremer, Niek Wijngaards and  Michele Jacobsen for many interesting discussions that led to many of the ideas for the reconceptualization of this course. Michele's help in particular in collecting the surveys and logs has been invaluable.

References

[GSS96] Goldberg, M.W., Salari, S. and Swoboda, P. (1996). World Wide Web Course Tool: An Environment for Building WWW-Based Courses. Computer Networks and ISDN Systems, 28. http://www.webct.com/we ebct/papers/p29/index.html

[Jacobsen98] Jacobsen, D. M. (1998). Adoption patterns of faculty who integrate computer technology for teaching and learning in higher education. Proceedings of the ED-MEDIA AND ED-TELECOM 98: World Conference on Educational Multimedia and Hypermedia & World Conference on Educational Telecommunications, Freiburg, Germany, June 20-25. http://www.acs s.ucalgary.ca/~dmjacobs/phd/phd-results.html

[Kristapiazzi98] Kristapiazzi, G. (1998). Compare Web Tools for Course Authoring. http://w www.geocities.com/Eureka/Gold/6012/compare_web_tools.htm

Eleni Stroulia

Department of Computer Science

University of Alberta

Edmonton, Alberta, T6G 2H1  Canada