Blended learning? Design and evaluation of a level 3 undergraduate fluvial geomorphology course

Abstract

This report describes the development, use and evaluation of a blended learning approach to delivery of Glacial and Fluvial Processes, a level 3 undergraduate module at Liverpool JMU. Students have responded positively to the introduction of new e-Learning resources (a specially designed web resource supported by funding from the FILTER project, Blackboard VLE) and a reading file of key journal papers. Although statistically insignificant due to the small sample size, the overall time spent on using module resources (web site, Blackboard VLE and reading file) correlates more highly with mean examination score than either lecture attendance or time spent reading journal papers only. Time spent reading journal papers correlates better with the percentage scored in an essay.

Both mean module mark and mean examination score were the highest for the four years that this module has been delivered, though the mean essay score was not. This evaluation far from proves that ‘blended learning’ improves examination performance as the analyses are hampered by the small sample size, but it does raise enough questions to warrant further investigation.

Introduction

A Level 3 option module entitled Glacial and Fluvial Processes has been delivered for the past four years as part of the BSc (Hons) in Outdoor & Environmental Education at Liverpool John Moores University (JMU). While this unique programme provides students with numerous opportunities to develop outdoor pursuits skills in climbing, kayaking, caving, skiing, winter mountaineering and other key areas of outdoor education (such as problem solving, adventurous activities, the John Muir Award, orienteering etc), an equally important aim of the programme is to develop students’ environmental understanding. Indeed, the course has been designed so that by selecting certain geography/environmental options at Level 2 and Level 3, and by undertaking a geographically based dissertation and work placement, students can end up with over 60% of their programme related to geography. Indeed, a number of recent graduates have proceeded to take up PGCE places in Geography and have gone into secondary Geography teaching.

In common with most GEES subjects in higher education, increasing class sizes, decreasing relative financial and time units per student, and enhanced IT power and availability have resulted in academics researching and evaluating alterative methods for delivering their subjects (e.g. Skinner, 2004; Staddon, 2004; Turney et al., 2004 in the July 2004 issue of Planet). This article reports on the move towards a form of ‘blended learning’ developed for OLFEO3205: Glacial and Fluvial Processes at Liverpool JMU.

Aims and objectives

This study aims to:

1. Describe the development and use of a blended learning approach to delivery of OLFEO3205: Glacial and Fluvial Processes, a level 3 undergraduate module at Liverpool JMU,

2. Present student evaluations of key aspects of this blended learning approach,

3. Make recommendations based on the experience of designing and evaluating this teaching resource.

e-Learning or Blended Learning ?

In 2004 both HEFCE [http://www.hefce.ac.uk/pubs/circlets/2004/cl09_04/ (accessed 29-9-04)] and DfES[http://www.dfes.gov.uk/elearningstrategy/online.cfm (accessed 28-9-04)] have published the responses to their UK wide e-learning strategy consultations. The HEFCE response states that there were a total of 114 responses from universities, colleges and other stakeholders and that:

“Respondents overwhelmingly requested a strategy that emphasises ‘blended’ approaches to learning and teaching — where e-learning via the web or other technologies is augmented by more traditional methods including classroom sessions, and the use of books and other resources — rather than wholly e-based learning”.

(quoted from http://www.hefce.ac.uk/pubs/circlets/2004/cl09_04/ accessed 29-9-04)

The concept of ‘blended learning’ has very much been used in the design and evolution of this module over four years. The module is delivered by two members of staff, one glacial processes (Professor, Dave Huddart) and one fluvial processes (Reader, Tim Stott). Table 1 summarises the delivery approaches used in this module. Styles 1 and 2 were used by the glacial expert, while styles 3–7 were used by the fluvial expert and are evaluated in detail as part of this report.

Table 1 Module delivery styles in OLFEO3205: Glacial and Fluvial Processes

1	Five ‘traditional lectures’ using slides, OHP, with detailed handbook on relevant topics (hard copy) and titles of follow-up recommended reading (largely journal papers)
2	One Glacial field day: describing and logging glacial sediments and evaluating models of sedimentation
3	Five lectures using PowerPoint incorporating images, diagrams, video clips, in-class tests.
4	Fluvial geomorphology Web Based Learning Resource (designed for fluvial part of module, FILTER project — see text for full description and URL)
5	Blackboard VLE: Five PowerPoint files of course lecture content, including in-class tests, mounted for students to access plus a past examination paper.
6	Blackboard VLE: Six PDF journal papers mounted for students to access.
7	Reading File: a collection of 20 relevant journal papers which students can borrow and return
8	Interactive GEOGCAL CD Rom on Catchment Systems: Introduction to Catchment Systems; Systems Framework; The Hydrological cycle;The Hydrograph; Catchment characteristics; Hydrograph Simulation.

New aspects of the module design

The FILTER project

In 2002–03 funding was obtained from FILTER (Focusing Images for Learning and Teaching - an Enriched Resource, based at the Institute for Learning and Research Technology (ILRT), University of Bristol) to develop an Image based Fluvial Geomorphology Learning Resource. The resulting web based level 3 undergraduate course in fluvial geomorphology is used in this module (see 4 in Table 1). It aims, via Units 1–4, to provide learners with a level of knowledge and understanding to work at undergraduate level 2/3 in Fluvial Geomorphology. Units 5–6 provide ideas, techniques and a suitable grounding for students wishing to undertake a dissertation study in the discipline. The resource assumes some basic understanding of the principles of hydrology and geomorphology. It includes 147 images (terrestrial and air photographs, maps, graphs, diagrams, GIS generated digital terrain models), VR Panorama movies and time-lapse aerial photography to act as a stimulus to introduce topics. Each topic then refers the learner to Essential Reading and if appropriate to Further Reading to provide the detailed knowledge and understanding required at this level. Readings are graded by their level of ‘friendliness’ to the newcomer to the discipline. Topics end with an assessment test

Planet readers are encouraged to access this resource, and numerous other resources free at: http://www.filter.ac.uk/database then choose Geography to go to the fluvial geomorphology resource developed for this Level 3 undergraduate module: http://www.filter.ac.uk/database/insightrecord.php?id=48

Blackboard Virtual Learning Environment (VLE)

In 2002 Liverpool JMU made the Blackboard VLE available via its network to all its staff and students. Since then there has been a concerted effort through the University’s Learning Development Unit to train staff, who in turn train students in its use. The University aims to have real content on Blackboard (Bb) for over 60% of its modules in the current academic year. The PowerPoint files used in the five fluvial lectures were mounted in the module content area of Bb for students to access and were made available before, during and after the five lecture sessions. In addition, and for the first time in 2004, six PDF files of key journal articles were mounted in Bb for students to access directly.

Bringing the Mountain to Mohamed: the Reading File

For the fluvial processes part of the module, a collection of 20 relevant journal articles, including several publications by the lecturer, were reproduced and made available to students before/during/after each of the lecture sessions. Care was taken to observe copyright regulations and staff at the Learning Resource Centre (LRC) assisted with this. Students were encouraged to sign out and borrow up to three papers (for one week duration maximum) for their own private study. Students were then asked to return the papers so that other students could then use them. All journal articles were available in the University’s three LRCs or as electronic journals to which the University subscribes. The idea of collecting together the papers into a reading file was to try to make the task of seeking out references both easier and less time-consuming for the students.

Other delivery methods

Other aspects of the delivery (see Table 1) included traditional slide and OHP lectures, a field day and use of an Interactive GEOGCAL CD Rom on Catchment Systems. However, these are not the focus of this report.

Evaluation method

The students’ evaluation of the FILTER web site (4 in Table 1), the Bb PowerPoint materials (5 and 6 in Table 1) and the Reading File (7 in Table 1) was undertaken by means of an optional end of module questionnaire which was completed by 12 of the 15 students registered on the module. The questionnaire was completed immediately after the module’s final examination, so students responded as individuals under examination conditions and did not discuss their responses with their peers or staff. This was not a deliberate decision, simply a convenient time to evaluate the module after it had been fully delivered and assessed and before students left University. One student did not attend the last three lectures or final examination due to personal problems.

Results and discussion

A self-reporting technique, as used in this study, can lead to erroneous interpretations (Denzin & Lincoln, 1994) concerned with students’ understanding of the question or statement, honesty in answering, and how well they have been able to remember facts about their studying and time spent using various learning resources. Clearly then, given the limitations associated with this kind of data collection technique, we can only proceed with caution and, as with all research, we make the assumption that the above sources of error apply equally to all respondents and so do not distort any patterns observed.

The evaluation first asked students to estimate how many times they had accessed the following resources offered: lectures;web-based learning resource, PowerPoint lecture files mounted in Blackboard VLE, reading file of journal papers, journal papers as PDF files mounted in Blackboard VLE. Figure 1 summarises the responses. In summary, 10 of the 15 students attended all five lecture sessions (Fig. 1A), with all students attending at least two.

This suggests that this aspect of the delivery was highly valued by the majority of students. Of the 12 students who completed the evaluation, three did not access the web based resource at all (Fig. 1B), despite it being highly recommended during the lectures. However, unlike for the GEOGCAL Catchment Systems CD Rom (Table 1, point 8), a computer room was not booked to demonstrate the web based resource, and perhaps seeing it demonstrated would have encouraged all students to use it.

The Blackboard mounted PowerPoint lecture resources (Fig. 1C) had greater use, with eight students accessing the resource seven or more times, and only one student not having used this at all. Of course, the same material (PowerPoint presentations and in-class tests) as is presented in the lecture sessions is made available here, so students will usually be mainly accessing it as an aid to note taking and/or revision, perhaps with some accessing them if they were unable to attend a lecture. Blackboard VLE was installed and made available at JMU in 2002, during this student cohort’s first year. It should be noted, therefore, that these students are, by now, quite used to accessing files and information by this means. Some of the more IT literate students may just access Bb only once per module, download the files they need and save them to CD or other memory device. They then take them away from the University network to view off-campus on their own home PC or lap top. This saves connecting to the JMU network from off-campus and perhaps incurring large telephone bills where large files need to be downloaded — an innovative and sensible use of the resource.

11 of the 12 students who responded claimed that they accessed the reading file, most accessing it 3–4times (Fig. 1D). This system was used once previously in their programme during a Level 2 module, but apart from that it is a relatively new feature of their learning. Students appreciate that loaning papers from this file is a less time consuming means of accessing the papers that searching for them in LRCs or on-line. The PDF files mounted in the Bb VLE, however, were used less than the reading file (Fig. 1E). This is probably because there were only six PDF journal papers mounted and these were also available in hard copy in the reading file. It appears that most students still prefer to loan a hard copy, and perhaps make their own photocopy of it, rather than read the article on-line, or print out from the PDF file.

Figure 2 illustrates the time spent using module resources. More than half of the respondents (7 out of 12) spent longer than 45 minutes using the FILTER web based resource (Fig. 2A), with 3 accessing it for 0–15 minutes, whereas 9 out of 12 spent longer than 45 minutes using Bb resources (Fig. 2B), with only one accessing it for 0–15 minutes. This suggests, not surprisingly, that the Bb resources were seen as more central and useful to their learning than the FILTER web based resource which was simply introduced to the students, but no time was spent accessing it during the lecture time. Students are much more familiar with Bb and what it has to offer, having been using it for 1–2 years. The evaluation asked students to use a Likert scale to give their response to a range of statements concerned with both the FILTER resource (Table 2) and Blackboard (Table 3).

Eight students spent more than 45 minutes reading journal articles loaned from the reading file (Fig. 2) with only one student spending 0–15 minutes. Less use was made of the PDF journal papers mounted in Bb (Fig. 2D) with five students spending 0–15 minutes and only 2 students spending more than 60 minutes. Use of the PDF files requires students to either read them on-line or on the computer screen, or to print them off (at their own expense). Journal papers from the file can be loaned with no charge. Some students loaned them and made their own photocopy for private study later if they didn’t have time to read and make notes on the paper during the loan period.

Figure 3 shows students’ self-reported reading of journal papers. This was an aspect of the level 3 learning which was emphasised as being important, and reinforced by provision of the reading file of journal papers. Fig. 3A shows that at one extreme three students loaned 7 or more papers from the file, while at the other one student loaned no papers. Eight students read and made notes on 7 or more papers (Fig. 3B) while at the other extreme again, one student reported making notes on 0 papers. Fig. 3C shows that students did not totally rely on papers loaned from the reading file — all students except one did seek out papers from other sources, and read them too (Fig. 3D).

As with the FILTER resource and Bb, the evaluation asked students to use a Likert scale to give their opinions in response to a range of statements concerned with the reading file of journal papers (Table 4 overleaf).

Table 2 Student responses to FILTER web based learning resource (n = 9)

	Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
I found accessing the resources too difficult so I gave up	0	0	0	5	4
The images are a useful way of stimulating my interest in this subject	5	7	3	0	0
The resource was easy to access	2	7	0	0	0
Having further reading is a good idea	0	4	4	1	0
There are too many images which only serve to confuse me	0	0	3	7	1
There were too many photographs and not enough diagrams to explain things	0	0	1	4	3
There is not enough writing, information or facts	1	2	2	4	0
I find further reading frustrating as I have to go to the LRC to find it	0	3	3	2	1
I attempted some of the assessment sections in the resource and found these useful to my learning/revision	0	6	3	0	0

Table 3 Student responses to Blackboard based learning resource (n=12)

	Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
Having all the lecture files here in module content 24/7 is a good idea	8	3	1	0	0
I had problems accessing the lecture PowerPoint files	0	3	3	4	2
I have made good use of Blackboard for this part of the module	2	7	1	2	0
I never use Blackboard as I went to the lectures and find all the information I need there	1	0	1	7	3
I didn’t get round to using Blackboard at all	1	1	0	2	8

Table 4 Student responses to reading file of journal papers (n=10)

	Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
Having a reading file with Journal articles is a good idea	10	2	0	0	0
The articles are up to date and meet my needs	4	5	1	0	0
The articles are too academic and I don’t understand what they are about	0	1	5	2	2
I haven’t had time to borrow any articles or do any reading at all	0	0	1	4	5
The articles in the file are out of date and not useful to me at all for this module	0	0	1	6	3
I was able to reference journal articles I’d read in my coursework essay	4	4	2	0	0
Having a reading file and expecting us to read stuff in it is just asking too much	0	0	1	6	3
I was able to reference journal articles I’d read in my exam answers.	0	2	4	2	2

Linking the learning with the assessment

The module is assessed by (a) a 1000 word essay from a choice of three titles provided (32% of overall mark), and (b) a 2 hour written examination (68% of overall mark) where students were required to answer three essay type questions from a choice of six (three fluvial, three glacial).

Table 4 shows the proportion of marks available in both the essay and examination for different aspects of answers. Students were provided with these details in their module handbook issued at the start of the module, and reminded of it at key points during the lecture sessions.

This marking scheme was introduced three years ago and experience showed that few students were able to demonstrate evidence for reading texts and journal papers and gaining knowledge beyond the key facts presented in lecture sessions. The reading file was introduced as a mean of assisting students to read and therefore be able to gain marks in section 4 of the marking scheme, particularly in the exam question

While none of the relationships in Fig. 4 are statistically significant because the sample size is too small, nevertheless, the trends observed provide a stimulus to investigate these relationships further. The trends are hardly surprising, and it is interesting to note the stronger relationship between the examination score v. overall time spent using three of the module resources (FILTER web site, Bb and reading file) (Fig. 4C) compared with the examination score v. time spent reading journal papers (Fig. 4B). This suggests that the alternative electronic module resources (FILTER web site and Bb) may be having some impact upon students’ attainment in examinations. However, Figs. 4E and 4F show the opposite trend where percentage scored in the essay has a stronger relationship with time spent reading journal papers (Fig. 4F) than with overall time spent using all module resources (Fig. 4E). This is most likely because the topics set for the essays, plus the marking system (Table 4) which awards 25% of the marks for citation of literature sources, means that students who read journal papers as preparation score more highly.

Table 4 Marking scheme for essay and examination questions.

	% in essay	% in exam questions
1. Structure, grammar, spelling	15	15
2. Content and evidence of knowledge and understanding of the question	40	50
3. Use of relevant illustrations (e.g. graphs, diagrams, photographic evidence, maps, charts, tables of data)	20	20
4. Citation of literature sources	25	15
This marking scheme was introduced three years ago and experience showed that few students were able to demonstrate evidence for reading texts and journal papers and gaining knowledge beyond the key facts presented in lecture sessions. The reading file was introduced as a mean of assisting students to read and therefore be able to gain marks in section 4 of the marking scheme, particularly in the exam questions.

Table 5 Past module mean marks for OLFEO3205: Glacial & Fluvial Processes compared with Final Degree Classifications.

Year	Mean Essay Mark (34%)	Mean Exam Mark (66%)	Mean Module Mark	Final Degree Classification Breakdown	Number of Students
				I	II.i	II.ii	III
2001	59	37	44	1	2	8	1	12
2002	40	38	39	1	4	9	3	17
2003	65	32	41	1	5	4	4	15
2004	55	45	47	0	3	8	3	15
2005	66	50	55	2	5	18	1	11

While it is clearly unwise to compare one cohort of students with another, it is nevertheless interesting to note the data presented in Table 5. Compared with other module marks for these same students at Level 3, marks for this module are consistently lower and the same external examiner has scrutinised these differences over the 4 years and agrees that the marking has been consistent and rigorous. Also, the staff and assessment system have remained constant.

In 2004, when the electronic learning resources (FILTER web resource, Bb) and reading file were introduced as new learning resources, it has been encouraging to note that the mean examination mark and the overall mean module mark were the highest they have been in all four years. The 2004 mean is 5.7% higher than the mean for the year 2001–2003 which is 41.3%. This could of course be due to the particular cohort of students, though comparison of their final degree classification profile with previous years would not suggest that they were a higher achieving cohort. We would prefer to suggest that the introduction of the additional learning resources (FILTER web site, Bb and reading file) may have helped students to improve their examination marks, though these have had no impact on the essay mark. However, to be able to prove this to be the case would, of course, be an extremely difficult, if not impossible, task !

Conclusions

Students studying a level 3 undergraduate module in Glacial and Fluvial Processes have responded positively to the introduction of new e-Learning resources (a specially designed web resource supported by funding from the FILTER project, Blackboard VLE) and a reading file of key journal papers.

Although statistically insignificant due to the small sample size, the overall time spent on using module resources (web site, Bb and reading file) correlates more highly with mean examination score than either lecture attendance or time spent reading journal papers only. Time spent reading journal papers correlates better with percentage scored in an essay.

Both mean module mark and mean examination score were the highest for the four years that this module has been delivered (though the mean essay score was not), even though final degree classification profile for this cohort of students is no better than that for the previous three cohorts.

This evaluation far from proves that ‘blended learning’ improves examination performance as the analyses are hampered by the small sample size, but it does raise enough questions to warrant further investigation.