Improving the Effectiveness and Efficiency of Teaching Large Classes: Development and Evaluation of a Novel e-Resource in Cancer Biology

Abstract

This paper describes the development and evaluation of a blended learning resource in the biosciences, created by combining online learning with formal face-face lectures and supported by formative assessments. In order to improve the effectiveness and efficiency of teaching large classes with mixed student cohorts, teaching was delivered through a variety of media which included three main components; (1) an interactive online tutorial, based on the cellular processes of DNA replication, damage and repair in relation to oncogenesis (2) formative assessment in the form of multiple choice questions to allow self evaluation and (3) small group follow-up workshops, to encourage deeper learning. The online tutorial was designed using Flash© software to help conceptualise complex cellular processes in time and space. It was supported by formative quizzes, references and printer-ready notes. Introduction of these resources in 2005 led to significant improvements in summative assessments across all student cohorts compared to scores from 2004. Students highly valued the usefulness of self-paced learning combined with supportive formative assessments which helped enhance the learning process. Teaching in small group workshops that followed on from the tutorials was also more effective, allowing a better interaction with the students, encouraging confidence and deeper learning among students. The efficiency of teaching was also improved with reduced assessment times and less pressure on institutional resources (availability of large lecture halls). This study therefore supports the use of blended learning as a means of improving both the effectiveness and efficiency of large group teaching.

Keywords:

• online teaching resource
• blended learning
• student diversity
• cancer biology

Introduction

Traditional ‘face-face’ lectures are frequently a passive experience for students and strategies that encourage active learning and deepen understanding are known to enhance the learning process (Bonwell and Eison, 1991). The use of active learning strategies is especially relevant for large classes and/or diverse student cohorts. Among the various approaches, the most popular and controversial is the use of online teaching resources (Newnham et al, 1999; Lemke and Ritter, 2000; Spellman, 2000). This ‘e-learning’ revolution tends to be primarily technology-led since innovations in e-learning are driven by innovations in information and communication technologies (Timms et al, 1997; Castleford, 1998). This has also led to widespread use of the technology, sometimes without the pedagogic support to enhance learning (Badge et al, 2005). Effective online learning resources can, however, have many benefits, including self-paced learning, access to the resource without any time or geographical constraints and an active learning experience for all students (Salmon, 2002). This paper reports on the outcomes of developing a teaching resource package, which included 1) an online tutorial aimed at developing a strong conceptual understanding of the complex cellular processes of DNA replication, damage and repair pathways in oncogenesis, 2) embedded self-evaluative formative assessment and 3) follow-up small-group workshops to encourage deeper learning. The objective of the project was to enhance both the efficiency and effectiveness of learning, teaching and assessment of this unit. Some of the anticipated outcomes include facilitating understanding and an active, self-paced learning among students, reduced assessments times and demand on teaching resources to support large-class teaching and most important, to enhance the student learning experience.

Method

The Cancer Biology Unit

The unit on cancer biology is taken by second year students pursuing a range of degrees in the Department of Biology & Biochemistry at the University of Bath. It was previously taught in a traditional 24x1 hour lecture format in large lecture halls. Some of the problems associated with teaching this unit were large class sizes (~180 students) combined with diverse student cohorts (drawn from Biology, Biochemistry, Molecular & Cellular Biology (MCB), Natural Science, Chemistry and Psychology programmes). The diversity of student backgrounds, combined with the large numbers involved, made it difficult to evaluate student learning, especially in the first couple of weeks of the module. In addition, there was a large amount of coursework assessment as part of this unit. In order to increase both the efficiency and effectiveness of learning, teaching and assessing this unit, the teaching resource package (as outlined above) was developed and introduced in October 2005.

Online Tutorial: The content of the online tutorial (www.bath.ac.uk/bio-sci/cbt - freely available for educational purposes) was based on the fundamental, but complex, mechanisms of DNA replication, damage and repair in relation to oncogenesis. This constituted approximately ten hours of lecture material. The design and layout of the resource was modelled on other effective sites such as DNAi (www.dnai.org). The resource was designed in two formats, Flash^© and html. Flash^© allowed the animation to be interactive, concise, maintain interest and have sufficient depth to enable conceptualisation of complex pathways in time and space whereas the structural framework and detailed information was provided on the printer-friendly html pages. The services of a professional multimedia designer (Simon Bos from Gravitywell) was the most effective and efficient way in developing the software (although not the cheapest). It took approximately three months for the designer to complete the website following a few consultancy sessions. The only input now needed is maintenance and updating the resource. The institutional teaching development fund helped finance this project. This resource also contained novel animation material, especially in eukaryotic replication and DNA repair. Full written instructions on how to access the website were e-mailed to students at the beginning of the unit and the site was accessible ‘on line’ at all hours from the start of the teaching semester. A text-only version of the resource was also e-mailed to students at the start, which contained detailed information, references and links to external websites. It could, therefore, be assumed that all students had access to an appropriate version of the notes. The printer-friendly text version was also linked to the tutorial and navigation footprints were introduced based on informal student feedback and advice from the e-learning team.

Self evaluative assessments: A series of multiple choice questions (MCQs) based upon the material contained in the resource were included on the website to allow students to both focus and test their learning.

Small group follow-up workshops: These were designed specifically for the four main degree cohorts (Biochemistry, MCB, Biology or Natural Sciences/other), to encourage deeper learning. Students were briefed on the details of the resource in the first lecture and were given three weeks before the first of the follow-up workshops. There were two workshops for each cohort, each of which included both ‘thought’ questions and ‘scenarios’ formulated on the tutorial, which were specifically designed based on the background knowledge of the four student cohorts. Students were encouraged to ask questions and discuss ideas during these workshops.

Results and Evaluation

The effectiveness of the teaching resource was assessed by monitoring student performance in a summative final written examination. All examination material was marked blind. The exam lasted two hours and was worth 80% of the unit mark. The examination was divided into two sections: a compulsory MCQ (20% of the unit based on content from the online tutorial) and single essay from a choice of three (60% of unit mark based on the rest of the unit content). A range of strategies including summative assessments, student feedback and informal teaching observations were used to evaluate the efficiency and effectiveness of the resource.

Summative assessments

Summative assessments for the year of introduction of the resource (2005–6) were compared to the control group from the previous year (2004–05). The assessments marks shown (Figure 1) are percentage scores for the MCQ (20% of the total for the unit), essay-based exam scores (60%), and practical component (20%), as well as the overall mark achieved by the four main student cohorts taking this unit. As seen in Fig 1, a marked improvement in MCQ scores was seen across all four cohorts compared to the previous year. The MCQ questions were identical in both years and thus, to ensure that students did not have access to these questions, MCQ sheets were collected at the end of the test. Interestingly, the traditional essay-based exam marks also improved across all cohorts (except the biochemists, whose average score was not significantly different), suggesting that the resource may have contributed towards a wider appreciation of the topic. Similarly, the overall marks were higher across all groups except the biochemistry cohort. The marks from the practical component however remained unchanged in both years across all cohorts.

Fig 1 Evaluation of summative assessments across four cohorts from the past two years. Data shows the % mean mark ± SEM across the 4 cohorts (MCB-Mol & Cell Biol; Natural Sciences & others; Biology and Biochemistry) in the 3 areas of summative assessment (MCQ, exam, practical, overall). The control groups were cohorts from 2004–05 (155 students) compared to 2005–06 (165 students). The distribution of students in each cohort was approximately 41–45 MCB; 24–29 Natural Sciences & others; 55–62 biologists and 26–28 biochemists

Feedback

The e-learning team at the University were initially consulted on ways to improve effectiveness of the resource. Based on their feedback, the site was enhanced by the creation of a printable web page in an html format. Subsequent modifications at this site were made to improve navigation and access, based on feedback from specifically-designed student questionnaires, and from B colleagues within the department. Surprisingly few students (2 out of 96) complained about accessing the tutorial from any location, suggesting that the software was compatible with most types of internet providers and also that students were comfortable with the use of the resource.

Student attitudes towards the provision of the teaching resource were determined towards the end of the unit by asking them to fill in an anonymous questionnaire. They were asked how useful they had found the online tutorial and animations, how useful they thought the resource had been compared to traditional lectures and their preferences on learning styles (web only, lectures only or both). Student responses to the website-related questionnaire (Table1) were collated around week 8 of the 11-week course (79% response rate). Of these, 90% of the students responded positively concerning the resource. The biggest advantages were perceived to be the opportunity for self-paced learning, use of animations and ‘24/7’ access provided by the resource. The use of graphics and animation also rated highly in the questionnaire. On the question of preferred choice of learning, the results were mixed. Approximately 22% of the students still preferred traditional lectures alone, 20% preferred the web-based learning alone whereas 58% liked a combination of the two.

Table 1 Results from the formative student feedback to website-related questionnaire taken on week 8 of an 11-week semester. The data is based on a 79% response to the questionnaire

Which method of learning do you prefer?
	Traditional lectures and note-taking only	22%
	Web-based learning only	20%
	Combination of both	58%
Did you find the web tutorial useful as a study resource?
	Very useful	91%
	No difference	5%
	Not useful at all	4%
How do you rate website navigation?
	Excellent	45%
	Very good	28%
	Adequate	26%
	Weak	1%
Did you find the workshops useful?
	Very useful	76%
	No difference	22%
	Not useful at all	2%
I found the self-evaluation MCQ very useful for revision
	yes	59%
	no	22%
	don’t know	19%
The use of animations in the tutorial were
	useful	92%
	somewhat useful	8%
	useless	0%

The format and content of the unit was disseminated to colleagues before implementation. Informal feedback was very encouraging and positive. The overlap of some of the subject areas covered by the resource also allows it to be used by other staff members for their teaching.

The outcomes of the project showed improved effectiveness and efficiency in teaching classes of over 180 students. The contents of the resource replaced material from approximately ten hours of traditional lectures. The workshops were divided between the four main student cohorts (approximately 40 students per cohort), with a contact time of two hours each. In effect, this meant that each cohort spent two hours in a lecture (instead of ten) with the rest spent in self learning. The total contact hours for the lecturer, therefore, amounted to eight. Although this is not a huge reduction in the total number of contact hours, the time was spent much more effectively in smaller group environments. This was evident from the marked improvements in the quantity and quality of questions that were asked during the workshops compared to the previous year’s teaching as well as the positive student feedback. Due to the student diversity, the workshops were tailored towards the specific degree cohorts. It allowed a better interaction with students, helping develop confidence among the quieter students and deeper understanding of the topic by the group on the whole. In addition, lesser formal lectures also meant reduced pressure on the use of limited teaching space. At the University of Bath, large lecture halls accommodating over 150 students are limited and tend to be overbooked. The smaller lecture rooms are more numerous and readily available. The reduced contact hours combined with use of smaller rooms helped reduce the demand for large lecture halls. The assessment times were also reduced by 50%, since the formal end-of-semester essay-based examination was reduced from two hours to one.

Discussion

Improvements in summative scores and comments made by students suggest that use of the online tutorial appears to have enhanced student learning and encouraged their interest in these topics, which were often found to be a bit dry by students in previous years.

The teaching resource was introduced to students in the first week of term, with the follow-up workshops scheduled two weeks later. This timing was deliberate to ensure that students had the opportunity to access and use the website before attending the workshops. Surveys showed that 63% of the class had done so before the first of the two workshops. Of these, 45% had done so out of curiosity about the online tutorial, whereas 55% were motivated to learn. As a result, the first of the two workshop sessions were spent revisiting the tutorial content and the second on deeper learning strategies using problems, ‘thought’ questions and concepts. Informal discussion with the students suggests that it was the motivated students that not only engaged with the learning but were also able to critically evaluate the contents and offer comments/suggestions. However, all students accessed the online tutorial in the few weeks before the exam, as part of their revision. Overall, ~86% of the students accessed the tutorial at least three times.

A vast majority of the students highly rated the online tutorial as an effective learning tool. However, more than half the group still preferred a combination of online learning along with formal lectures. This preference is similar to other reports from students studying subjects such as physiotherapy (Dewhurst et al, 2000) and medicine (Vogel and Wood, 2002). It reinforces the concept that, despite the widespread use of online learning resources, students still prefer to have the subject taught by people rather than computers alone and value formal contact time with their lecturers. Factors that may influence these preferences include financial (’lecturer’s get paid for lecturing’) or other learning issues such as a preference for passive learning (’easier to attend lectures than to learn myself’) or learning approaches (’like to make notes during lectures’ or ‘prefer to listen to lectures’).

The effectiveness of the resource as a learning tool is clearly evident from the improvements in summative MCQ scores, related to the online tutorial content. In addition, the improved scores in the essay component of the unit, which was based on content delivered as traditional lectures on topics that were indirectly related to the material covered in the online tutorial, suggests that student learning is also enhanced using computer-assisted learning (CAL) strategies. This improvement is unlikely to be due simply to the 2005 cohorts of students being more able than those in the previous years for several reasons (a) the entry level scores have been the same for at least the past six years (b) the average first year scores for the two intakes have been broadly similar (63.4 for the 2004 intake and 64.1 for the 2005 intake) and (c) there were no marked improvements in the practical component of the unit in both years across all cohorts. Increasing use of interactive CAL strategies in textbooks (as accompanying CD-ROMs) as well as evidence in the literature (Salmon, 2002; Mayer and Moreno, 2002; Pavey and Garland, 2004) further testifies to the popularity of CAL as a pedagogic tool. However, an exclusive use of CAL packages may not be as effective (Dewhurst et al, 2000; Pinckney et al, 2001). Dewhurst et al (2000) compared the exam marks for two questions, one of which covered a topic that had been taught entirely through a computer tutorial and the other which had been covered in traditional lectures and found mean marks to be equivalent. The reasons for the lack of effectiveness of a CAL package as the only learning tool are not known but it suggests that the package needs to be carefully designed to take the maturity of the students as learners into account. The general consensus is that computer-assisted learning packages are at least as effective in imparting information as traditional lectures and that students value the opportunity to support and reinforce their learning through interactive resources and self-assessment quizzes. Data from the current study also confirms previous research which suggest that most students use self-assessment quizzes as part of their revision programme towards the end of the semester (Gunn and Pitt 2003), rather than to reinforce learning.

In conclusion, the use of CAL combined with face-face contact via workshops was effective in enhancing student learning especially in large classes. Teaching efficiency is also enhanced by improving teaching quality (with reduced contact and assessment times) and reducing the demand on resources placed by large class teaching.

Acknowledgements:

Simon Bos, multimedia designer from Gravitywell (www.gravitywell.co.uk) created the online tutorial design, layout and animation. Dr. Derek Morrison, Lisa Williams and Graham Blacker from the e-learning team at the University of Bath as well as the students and tutees doing the unit provided invaluable feedback on the resource. The project was funded by the University of Bath Teaching Development Fund.