The effectiveness of computer-based teaching packages in supporting student learning of parasitology

Abstract

The usefulness of providing students with information via electronic notes, a website and a self-assessment quiz was assessed in three ways. Students’ perceptions of each package were gauged through questionnaires; their patterns of use of the electronic support was monitored via the computer system and the effect of provision of these materials on performance was measured by marks in the end of module examination. The results clearly showed that computer-assisted learning packages are at least as effective in imparting information as traditional lectures and that students value the opportunity to reinforce their learning through interactive lecture notes and self-assessment quizzes. However, students definitely expressed the opinion that they would not like to have too much of their lecture time replaced by computer packages and they seemed to require the option of ‘downloading’ the information onto a piece of paper for individual use. Therefore, it seems that the teachers who are seeking new ways to teach students must avoid overuse of computer-based packages and use other forms of student-directed learning, such as problem-based tutorials, as well. Also, they should include at least as much contact time with students as in a formal lecture programme if they are to support learning effectively.

Keywords:

• Electronic notes
• website
• self-assessment quiz
• performance
• parasitology

Introduction

The role that computers play in the teaching of medical and biological subjects in higher education has increased dramatically in recent years. The introduction of networked computers into the academic environment has meant that communication and searching for information through the internet have become important skills for both students and teachers to learn (Ward et al., 2001). The students of the 21^st century are very comfortable with the use of computers since they will have been using them in an educational, as well as social context, since primary school. Teachers need to rise to the expectations of such students and make use of the opportunities that computer technology affords.

It is important for academic staff to be aware of the possible uses of computer technology and to be confident in introducing it to their teaching. The technology is now widely accessible to lecturers without the need for advanced computer skills. Such technology allows them, for example, to put notes on the networked system, introduce self-assessment exercises or observe external specialist web-sites (in order to recommend them to their students). Computer facilities are usually available to students within the university or college site outside of the normal working day and it is generally possible to access the networked site for most academic institutions externally, which allows students to ‘log-on’ via the internet, for example, from home or in an internet café (e.g. Ward et al., 2001). In addition, a Virtual Learning Environment (VLE), such as Blackboard (www.blackboard.com) or WebCT (www.webct.com) can be used to provide administrative as well as academic support for students.

There are a variety of ways that Computer Assisted Learning (CAL) has been used support students (Race et al., 1999; Ward et al., 2001). The simplest package involves a lecturer providing their notes electronically, either at the beginning of a lecture series or after each timetabled session, in place of ‘handouts’. In electronic form, the notes can be complemented by illustrations (Ryan et al., 2000; Ward et al., 2001) and links to relevant web sites (Cann and Pawley, 1999; McEhlone, 1999). Students apparently value this type of resource, since it allows them to be confident that they have understood the salient points of a lecture series, helps them to become familiar with technical terms (e.g. species’ names) and assists them in navigation through the plethora of information available on the internet (Cann and Pawley, 1999).

Multimedia CAL packages designed to replace lectures are also being introduced into medical and biological science teaching (e.g. Dewhurst et al., 2000; Ryan et al. 2000; Koner et al., 2001). They can be attractive to teachers for both logistical and educational reasons. A CAL package in place of a series of lectures to a large class solves the problem of finding a suitable lecture room in a congested university timetable. In one study, in which a package was evaluated by several cohorts of students, time was allocated for them to use the computer rooms in the first year but subsequently, students were expected to organise suitable times to use the computer package themselves (Dewhurst and Williams, 1998; Dewhurst et al., 2000). By appealing to a range of learning styles, a well-designed package should also empower students to become independent, ‘active’ learners, rather than passive recipients of information (Ramsden, 1999). However, there is no strong evidence that students who are taught through CAL, rather than conventional lectures, perform better in assessment (e.g. Dewhurst et al., 2000; Koner et al., 2001) and when asked, students are not enthusiastic about replacing their lecturers with computers (Dewhurst et al., 2000; Vogel and Wood, 2002; Gunn and Pitt, unpublished data).

Another area of biological science teaching that can be supported by computers is laboratory practicals (Miles, 1999; Hughes, 2001; Ryan et al., 2000; Riley et al., 2002). Computer simulated practicals can overcome the problems of the use of live animals (Miles, 1999) or the scarcity of clinical specimens (Riley et al., 2002). Computer packages can also be valuable in teaching diagnostic and treatment skills to medical and veterinary students, since the ‘virtual’ patient cannot be harmed (Ryan et al., 2000).

It is now feasible to devise electronic multiple choice tests, with the aid of programmes such as Question Mark (www.questionmark.com). These can be used for summative assessment but the practical problems of identifying the candidate through their ‘log in’ details and ensuring that there are sufficient terminals in an examination room often make this harder than setting and marking a paper examination (Ward et al., 2001). However, the use of computer-based multiple choice tests as formative or self-assessment has been widely reported in the teaching of medical and biological sciences (e.g. Cann and Pawley, 1999; Sly and Rennie, 1999; Pinckney et al., 2001, Steele et al., 2002). These would be expected to help students test their factual knowledge and familiarise themselves with technical terms and key scientific theories (Ward et al., 2001). It is much easier to include clear illustrations (photographs of patients with particular disease symptoms, microscope slides of stained tissue containing a particular organism, electron micrographs, etc) in a computer-based rather than a paper-based test. It is possible to design a programme that selects a random set of questions each time the test is accessed so that, by repeated use of the test, the student can assess their learning thoroughly (instead of their ability to recall answers in the correct order!). Students are reported to find computer-based self-assessment tests useful, particularly for revision (Steele et al., 2002). There appear to have been only a few studies of the effect of these tests on assessment performance and they tend to show that improvement is small (e.g. Pinckney et al., 2001).

Through the internet and VLEs in academic institutions, teachers of parasitology could make some or all of these applications of CAL available to their students. There are logistical advantages to using computer packages in some situations and theoretically they should enhance student learning by encouraging each individual to work in a way and at a pace that suits them. This ought to promote deep, reflective learning (Ramsden, 1999), allowing students to understand the material and apply it subsequently - not only to pass assessments but in their professional life.

Despite the wide availability of CAL packages and high expectations from some students and teachers, there appears to be a lack of studies clearly evaluating their effect on student learning (Vogel and Brown, 2002). In this study, three types of CAL package were gradually introduced, over three academic years, into a Level 3 Parasitology undergraduate module and the value of each of these packages was assessed in three ways:

1. By assessing the students’ perceptions of each package as a learning experience.

2. By monitoring students’ use of the learning materials.

3. By measuring the effect of the availability of CAL within the module on student performance.

Materials and methods

The Parasitology module

The Parasitology module was introduced in the 1999/2000 academic year as a final year option for honours students pursuing a range of degree routes in the Schools of Biomolecular Sciences and Biological and Earth Sciences at Liverpool John Moores University. The mean number of students taking the module over 4 academic years was 32 (SD 8.5). There are a number of problems involved in teaching parasitology, including the difficulty of obtaining suitable specimens to use in practical classes and to illustrate lectures; the fact that parasite life cycles can be complex; and the lack of ‘common names’ for most of the organisms studied. Imagination and enthusiasm are required from the lecturers to avoid the subject becoming tedious for students and consequently difficult to learn.

1. Delivery of electronic notes

During 1999/2000 and 2000/2001, electronic lecture notes covering 13 lectures (43% of lectures delivered during the module) were provided. In 2001/2002 and 2002/2003, the two lectures covering myiasis were replaced by material delivered via an interactive CD (see 2 below) and an additional lecture, on malaria, was included for which electronic notes were also provided. Therefore, the total number of lectures for which electronic provision of notes was available became 40%, with an additional 6.7% delivered via the website. A text-only version of the lecture notes was e-mailed to the students before the start of the module and a version containing figures, diagrams, photographs etc could also be accessed by the students throughout the semester and downloaded from the university mainframe computer. The notes were a complete record of the information delivered by two lecturers and included extensive references and links to external websites. It could, therefore, be assumed that all students had a full and accurate version of the notes. Other lecturers on the module did not provide complete records of their lecture material to students in electronic format, although some gave printed handouts.

2. Delivery of lecture material via a website

An interactive CD was developed to teach an aspect of parasitology known as myiasis — the infestation of living vertebrates with the larvae of Diptera (true flies). It included numerous photographs, diagrams, computer animations and video clips and was fully interactive. It was introduced as a replacement for lectures on myiasis in the 2001/02 academic year. For logistical reasons, the students were provided access to the material via a website on the university’s computer network. Full written instructions on how to access the website were given to students at the beginning of the module. A series of questions based upon the material contained in the CD were included on the website to allow students to both focus and test their learning. Students who successfully answered the questions would have obtained information equivalent to two ‘normal’ lectures. The website provided the students with far more information than this. Students who browsed through the whole website and / or consulted the references or other websites mentioned would have been able to read extensively around the subject. Time was allocated on the timetable for students to access the material, although the site was accessible ‘on line’ at all hours from the start of the module. The website was not designed for ease of downloading and printing off of information: this was to encourage students to interact with the information and to make their own notes. Near the end of the module, a revision session on myiasis was given to further direct students towards the salient points and answer any queries or address problems arising. The student questionnaire was distributed at this time.

3. Computer-based self-assessment quiz

A computer-based self-assessment quiz was placed on the university computer network in March, 2001, and subsequently remained available at all times. The quiz covered all aspects of the module and was specifically designed to improve the students’ knowledge and recall of factual information. ‘Ideal’ answers were accessible by clicking on a button below each question. Students were repeatedly encouraged to use the quiz as part of their revision programme and suggestions on how to do this were included in the module handbook. A counter was included in the quiz but owing to the university’s rules of confidentiality, it was not possible to track the use of the quiz by individual students.

4. Student attitudes to the provision of teaching material in website format

From the academic year 2000/01 onwards, student attitudes towards the provision of information in electronic format were determined by asking them to fill in an anonymous questionnaire towards the end of the module. They were asked how useful they had found the electronic notes and the links, how many times they had accessed the website, how useful they thought the interactive CD had been compared to traditional lectures, whether reinforcement with a summary lecture was important, and how many more of the 30 lectures on the module could usefully be replaced by similar websites.

5. Student performance

The effectiveness of the different delivery styles was assessed by monitoring student performance in the Short Answer Questions (SAQs) and essay sections of the final written examination. All examination material was marked blind and moderated by a second marker. The exam lasted 3 hours and was worth either 60% (1999/2000 to 2001/2002) or 50% (2002/2003) of the module mark. The examination was divided into three sections: a compulsory interpretation question, a short answer section consisting of 8 questions from a choice of 10 and an essay section requiring a single essay from a choice of 4 (1999/2000) or 5 (2000/01 to 2002/03).

Results

Students’ reported use of the myiasis CD via the website in academic years 2001/02 and 2002/03 and their attitudes towards the programme as a teaching tool are shown in Table 1. This shows that students in the 2002/03 cohort stated that they accessed the website more frequently than those in the previous year. The majority of students in both years rated the website as equally valuable or better than ‘traditional’ lectures and 12% of those returning a questionnaire in 2002/03 rated it ‘much better’. However, over 80% in each cohort indicated that they felt that a summary lecture was needed to reinforce the material presented on the CD. Most students considered that it would not be desirable to have more than 3 further lectures replaced with a similar interactive website and no one stated that they wanted to see the majority of the course delivered in this fashion.

Table 1 Students’ stated attitudes to the myiasis interactive CD via website

Question		2001/02	2002/03
	N=35	N=27
26% questionnaire return	63% questionnaire return
How many times did you use the myiasis programme on the website?	Rating
	0	33%	6%
1	44%	29%
2	0	35%
3	22%	6%
4	0	18%
5	0	0
6	0	6%
How would you rate the myiasis programme compared to normal lectures?
	Much worse	0	0
Worse	0	12%
Equal value	33%	41%
Better	67%	29%
Much better	0	12%
Is reinforcement of the teaching material necessary?
	Yes	83%	94%
No	17%	6%
How many more of the lectures on the Parasitology module would you like to see replaced by similar websites?
	0	17%	29%
1–3	50%	41%
4–6	17%	24%
7–9	17%	0
10–12	0	0
>12	0	0

The influence of the provision of electronic support for students on choice of examination questions and performance in the Parasitology module is shown in Tables 2 and 3. Table 2 indicates students’ attempts of compulsory short answer and choice of essay questions in relation to the provision of electronic notes or the website to underpin the topic. This shows more than ¾ of the class in all years attempted short answer questions on topics for which the lecturer (AG or SP) had provided electronic lecture notes, compared with less than 50% in some cases for questions set by other members of the teaching team. A short answer question on myiasis was set in the 2000/01 year (before the replacement of lectures with the web-based teaching package) and in all subsequent years and the majority of exam candidates attempted it. Essay questions set by AG were the most popular in all years (Table 2).

Table 2 Influence of CAL on choice of examination questions

		SAQs ^1 : % student attempting question
Lecturer	CAL?	No. of questions	1999/ 2000 N=24	2000/ 01 N=43	2001/ 02 N=35	2002/ 03 N=27
AG	Yes	5	100	96.7	81.3	89.6
SP	Yes	1			78.1	88.9
RS	No	2	100	51.1	62.5	57.4
SC	No	1	100	81.4	46.9	77.8
MB	No	1 or 2	100	41.9	34.4	66.7
		Essays^2 : % students attempting question
Lecturer	CAL?	No. of questions	1999/ 2000 N=24	2000/ 01 N=43	2001/ 02 N=35	2002/ 03 N=27
AG	Yes	2	90.5	72.1	90.6	55.6
SP	Yes	0
RS	No	1	9.5	20.9	3.1	11.1
SC	No	1	0	7.0	6.3	22.2
MB	No	0 or 1	-	0	0	11.1

¹ No choice of SAQ was offered in 1999/2000; in subsequent years, students were asked to attempt 8 questions from 10

² Students were asked to attempt 1 essay from 4 in 1999/2000; in subsequent years, the choice was 1 from 5.

Where a lecturer set more than one question the average percentage of students attempting those questions is presented.

N = number of students sitting the exam.

Table 3 shows the mean mark for SAQ and essay questions set by each members of the teaching team. To investigate the influence of the provision of electronic support on performance the marks gained in questions from each staff member were compared with those set by AG, who had provided electronic lecture notes for all sessions. The results indicate that students achieved higher average marks for questions on topics that had been reinforced through electronic notes and self assessment quizzes, with the exception of the 2002/03 cohort. There was also no significant difference in their performance in the myiasis question in the first year of the use of the website compared with the previous year, nor in comparison to other questions regarding material that had been delivered through traditional lectures by the same lecturer (AG). However, students did perform significantly worse on the myiasis question in 2002/03. All the exam questions were considered to be of equal difficulty: they were vetted by the module leader (AG), a School committee and the external examiner. The exam answers were all double marked using the ‘ideal answers’ provided by the setters and no evidence of harsh or lenient marking was found. Students were aware of the distribution of exam questions between members of the teaching team. They might, therefore, have been expected to concentrate their revision on those subjects delivered by a lecturer providing few lectures on the basis that ‘question spotting’ would be easier. This, however, was found not to be the case.

Table 3 Influence of CAL on performance in exams

		SAQs: Mean mark per question
Lecturer	CAL?	1999/00 N=24	2000/01 N=43	2001/02 N=35	2002/03 N=27
AG	Yes	5.8	7.0	7.0	6.3
AG: Myiasis	Yes	-	6.9	7.0	5.3^$
SP	Yes	-	-	7.4	6.5
RS	No	2.2**	5.5*	3.7*	6.1
SC	No	4.9	3.5*	4.3*	5.7
MB	No	2.1**	2.6**	3.9*	5.3
		Essays: Mean mark per question
Lecturer	CAL?	1999/00 N=24	2000/01 N=43	2001/02 N=35	2002/03 N=27
AG	Yes	55.8	55.7	57.2	48.8
SP	Yes	-	-	-	-
RS	No	52.5	42.8	57.0	51.7
SC	No	na ^1	46.7	10.0**	57.0
MB	No	-	na	na	35.0

¹ na = no attempts at question set by this lecturer

* Mann-Whitney U-test indicates significant difference between mean score for questions set by this lecturer, compared to mean mark for questions set by AG, p<0.05.

** Mann -Whitney U-test indicates significant difference between mean score for questions set by this lecturer, compared to mean mark for questions set by AG, p<0.01.

^$ Mann Whitney U-test indicates significant difference between mean mark for Myiasis SAQ and other SAQs set by AG.

N = number of students sitting the exam.

Short answer questions were marked out of ten. Essay questions were marked as a percentage.

The self-assessment quiz was first made available to students in the middle of the module during 2001 and remained permanently accessible thereafter. The quiz covered all aspects of the module. Students were repeatedly encouraged to use the quiz as part of their revision Figs. 1–3 present the tracking data for the self assessment quiz, showing the weeks of the semester (Fig. 1), the days of the week (Fig. 2) and the times of day (Fig. 3) that students used the quiz. The ‘academic weeks’ referred to in Fig. 1 includes both holiday and revision periods and the results have been adjusted so that week 18 is equivalent to the week in which the examination took place. This was done because of variations in the length of the semester and to facilitate a more direct comparison between years.

Fig. 1 Percentages of each student cohort using the computer-based self-assessment quiz each week. Data has been adjusted so that week 18 is equivalent to the week in which the examination took place.

Fig. 1 shows that students used the quiz throughout the module but did not do so in any numbers until the three weeks before the exam. The data indicates that many students used the quiz only once or twice in 2001 and 2002 but use was more frequent in 2003. Unfortunately, the counter records did not allow the identification of individual students or of students who logged on several times on different days. This data would suggest that in 2001 and 2002 most students used the quiz one or two times but this increased to two or three times in 2003. These figures may be slight underestimates because although the quiz was designed to be used on-line a number of students complained of the difficulty in printing off both the questions and answers.

In 2003, the teaching day changed from Wednesday to Friday but this appeared to make little difference to the pattern of use, with most students using the quiz between Tuesday and Thursday (Fig. 2). Relatively few students used the quiz at weekends.

Fig. 2 Percentages of ‘log-on’ events for the computer-based self-assessment quiz for each day of the week.

From Fig. 3 it can be seen that the majority of students used the quiz during normal working hours with use peaking between 10.00 and 16.00. No students used the quiz between midnight and 8 in the morning and relatively few used it during the evening.

Fig. 3 Percentages of ‘log-on’ events for the computer-based self-assessment quiz throughout the day. There were no ‘log-ons’ between midnight and 8am.

Discussion

The provision of support for students through CAL appears to have helped their learning and encouraged their interest in the topics taught. This is clear from their favourable reactions to the interactive CD teaching package and the fact that students were more likely to choose to attempt examination questions set by the two lectures who had made electronic, interactive lecture notes available.

The data from the two years that the myiasis CD has been used indicate that it is an effective, but limited teaching tool. In the first year that it was included in the module, a third of students stated that they had not accessed the site at all by the time the questionnaire was distributed (during the revision ‘lecture’ on myiasis), whereas this figure fell to 6% in the second year (Table 1). However, the questionnaires were distributed in week 9 (out of 12) of the teaching timetable, which was 4 weeks before the examination date. This date was chosen to check whether students had incorporated accessing the site into their lecture programme or were ignoring it until starting their revision. Monitoring of the use of other forms of computer-based support, such as the self assessment quiz, among this group has shown that many students tended to access the packages only in the few weeks before the exam, as part of their revision (see below). It may be that some students who had not used the CD at the time the questionnaire was given out, nevertheless did use it later in the semester. No student stated that they had used the CD more than three times in the 2001/02 cohort, although in 2002/03 some students used it 6 times. Although the questionnaire was anonymous, informal talks with students leads us to believe that those who accessed the quiz the most times were the most highly motivated among the group. No records are available of how long individual students spent on the site each time that they accessed it. It had been anticipated that use would have been greater than this, as there were a variety of sections, including annotated photographs of pathogens and infected animals, moving illustrations and video clips, which would have warranted repeated inspection. The majority of students who expressed an opinion on the value of the CD as a learning tool rated it equal to, or better than, traditional lectures (Table 1). However, they were virtually unanimous in their desire for a session with the lecturer to reinforce the material. Also, in both cohorts, most students stated that they would not want more than three additional lectures replaced by a CAL package (Table 1). In their expressed desire to have the subject taught by people rather than computers and maintain formal contact time with their lecturers, these students are similar to others studying different subjects such as physiotherapy (Dewhurst et al., 2000) and medicine (Vogel and Wood, 2002) - in the UK and in other parts of the world (e.g. Koner et al., (2001) reporting on students in Nepal, and Henly and Reid, (2001) whose study was in Australia).

Only two cohorts of Parasitology students have used the myiasis CD, so its impact on learning is hard to evaluate. The average mark for the short answer examination question on this topic in the academic year 2000/01 — the year before the CD was introduced — was actually higher than the overall mean for similar questions set by the same lecturer (AG). The following cohort (2001/02), who were the first to use the CAL package, achieved similar marks in the myiasis question as their colleagues had in the previous year. Also, the mean mark for the myiasis question that year was the same as for other short answer questions set by AG. Previous studies, such as that reported by Dewhurst et al., 2000), support these findings. They also suggest that students are not deterred from attempting exam questions for which the material has been delivered entirely by a CAL package and that overall performance in summative assessment is not usually affected. For example, Dewhurst et al. (2000) compared the exam marks for two questions, one of which covered a topic that had been taught entirely through a computer tutorial and the other which had been covered in traditional lectures. They found mean marks to be equivalent. Pinckney et al. (2001) developed a computer-based tutorial to underpin, rather than replace, a series of lectures and practicals on the topic of ticks. Students were given unlimited access to the package but use was voluntary. Over four academic years, marks in the final assessment were only significantly higher in the group who used the tutorial for one cohort while for the other three years, marks were almost the same (Pinckney et al., 2001). In the present study, students in the most recent group (2002/03), fared relatively badly in the myiasis examination question, despite reporting greater use of the CD. However, their marks for other parts of the exam also compared unfavourably with students from previous years. The reasons for this are unknown and further data is required from several more cohorts of students using the CD for a more complete evaluation. However, the results presented here and previous reports suggest that students do not perform any worse in assessment when they have been taught by a CAL package, suggesting that CAL packages can be an effective teaching tool. However, the package needs to be carefully designed to take the maturity of the students as learners into account. For example, Devitt and Palmer (1999) tested three forms of a CAL package designed to teach anatomy and physiology to level 2 medical students and they found that the students who had used a didactic version, written in the style of a text book, performed better in a subsequent test of knowledge than those who had been given a problem-based learning, or a free text response, style CAL package. Strategic use of an interactive CD such as the one developed and evaluated in this study, could be useful in courses where the staff : student ratio is low or for teaching specialised topics in lieu of using an external lecturer.

The provision of electronic, interactive lecture notes appears to be a useful tool in supporting student learning. Considering the four cohorts of students, they were more likely to choose examination questions on topics which had been reinforced in this way (Table 2) and achieve higher marks (Table 3). Other authors have found that students enjoy investigating relevant external websites, but value the guidance from their tutor through an internal webpage, giving computer based links. (e.g. Cann and Pawley, 1999; McEhlone, 1999). Students on the Parasitology module were invited to suggest websites that they have found which could be included with the lecture notes, thus contributing to the site. For level 3 students, the finding and evaluation of suitable sites is a useful skill and it would be expected to encourage interest in the subject. Another feature of the electronic lecture notes was the inclusion of diagrams and photographs of parasites and the diseases they cause. The incorporation of this type of illustration can be very helpful for teaching practical identification and diagnosis in Parasitology and other clinical sciences (e.g. Ryan et al., 2000), especially when ‘live’ specimens of an infectious organism are hard to obtain or the disease is rare.

Although it is not possible to monitor individual students’ use of this type of computer based support on the Liverpool John Moores University network, the data about the overall patterns of use suggest that most students used the self-assessment quiz as part of their revision programme towards the end of the semester (Fig. 1), rather than to reinforce learning as the module progressed. This is likely to be because they were told it would cover all parts of the semester and may not have felt confident about testing themselves before they had finished the taught part of the module. Despite the opportunities afforded to them by the University to use networked PCs until later in the evenings and at weekends, most did not tend to use the quiz outside ‘office hours’ (Figs. 2 and 3). However it was interesting that some students complained that they had experienced difficulty in printing off the questions and answers, so that they could study at home. The request to do this had not been anticipated, but the conversion by students of a formative computer-based assessment package into a paper version to use in their own time has been reported previously (e.g. Thelwall, 2000). This observation and the fact that students mostly used the package in the middle of the day and in midweek suggest that there is a limit to the amount of academic work that students are comfortable with doing by computer.

Acknowledgments

We would like to thank Dr Graham Triggs for his assistance in preparing the myiasis CD-ROM used in this investigation.