Transforming the student experience at a distance: designing for collaborative online learning

Abstract

Distance learning programmes allow greater flexibility of learning. They are often necessary for those who wish to enhance their professional development and/or gain higher-level qualifications while simultaneously continuing to work, especially where appropriate courses of study are not offered locally. However, the isolation and lack of feedback experienced by off-campus learners are key factors in the low retention rates common to many programmes. Dependence on freelance, off-campus tutoring staff brings challenges for managing change. While there is no single solution for these problems, the student experience can be significantly improved by weaving the development of an online community into the fabric of the course of study and addressing challenges such as the relevance of curricula (including assessment and feedback procedures) through the use of appropriate technologies.

This article describes the redevelopment of a series of distance learning modules through the application of established educational principles. It explains the need for and approach to change and documents how learning technologies have been applied to combat student and tutor isolation, improve the quality, quantity and variety of feedback sources and increase student engagement with feedback.

The lessons learned throughout this redevelopment project will be of interest for educators developing or redeveloping online blended or distance learning programmes, particularly in Engineering and/or Built Environment subject areas.

Context

The modules discussed here form the core component of a part-time distance learning Masters programme in the department of Architecture and Civil Engineering. Prior to the start of redevelopment, the majority of which took place from August 2007 to February 2009, the modules were in the form of paper ‘workfiles’ comprising content organised in sections, individual reflective exercises and essay-style assignment tasks. Each module also incorporated a written examination.

Driving force for change

Although the programme had been attracting applications and producing graduates for several years, attrition rates were high, with many students withdrawing or deferring during the first six months of the three-year programme.

High attrition rates have historically been common among distance learning programmes (Diaz, 2002; Parker, 1999). A large number of studies based on surveys of withdrawn students (Minich, 1996; Powell et al., 1990, among others) conclude that external factors such as family responsibilities or changes in employment circumstances are largely responsible for withdrawal from distance learning courses. Clearly these factors are outside institutional control. However, when we asked our students (through an anonymous online survey) at the outset of this project what they found most challenging or demotivating about distance learning, they didn’t focus on external factors but spoke about feeling isolated, wanting more formative feedback and feeling insufficiently prepared for academic work.

Perhaps students do not feel that there is any point, having already withdrawn from a programme, in citing insufficient support or poor course design? Or perhaps, as Brown (1996, citing Robinson, 1981 and Simpson, 1993) states: ‘off-campus students tend to blame themselves for problems connected to their enrolment.’ Brown presents extensive evidence in support of internal factors (such as insufficient support from tutors and difficulties in contacting tutors) having a more significant role to play in non-completion of distance learning programmes. Assuming this is the case, a question is raised as to what is different about the experience of students who do succeed.

According to Thompson (1998), successful distance learners exhibit the following traits:

• highly motivated

• independent

• active learners

• good organisational and time management skills

• the self-discipline to study without external reminders

• able to adapt to new learning environments.

This list of highly desirable characteristics is in itself an indirect explanation for high attrition rates in distance learning. The majority of learners require encouragement, reassurance and scaffolding to succeed and without this only the most self-sufficient of distance learners will survive.

Thompson’s is not the only commentary that focuses on the characteristics of the learner. There have been many attempts (e.g. Parker, 1999) to develop a means of predicting an off-campus student’s chances of success or failure, with a view to targeting additional support at those who need it. Interestingly, few of these studies present the argument that it is better to provide a supportive learning environment for all distance learners from the outset.

By referring to established theory on what promotes learning we can gain insight into why distance learning often fails and the principles that should be addressed when designing it. There are several renowned summaries of what constitutes a good learning experience, for example Ramsden (1992), Mentkowski (2000) and Bransford et al. (1999). Chickering and Gamson’s Seven Principles (1987) refer specifically to undergraduate education, but are arguably just as relevant to postgraduate distance learners. Table 1 presents the challenges of meeting these principles in traditional distance education.

It is worth noting that the rapid growth of online distance learning has not had a significant impact on retention, with online courses also experiencing high drop-out rates (Diaz, 2002). It is important for course directors and learning designers to bear in mind that, regardless of the technologies used, the student experience will not be improved unless the principles listed in Table 1 are addressed.

Mandate for change

We set out to increase student retention by addressing some of the challenges listed in Table 1 that had been cited most frequently by our own students in the initial survey, tackling isolation, opening and enhancing feedback channels and providing the necessary support for students with little experience of academic study.

Approach to change

The approach we took to redevelopment of the modules is set out in Figure 1 using an adaptation of Everard, Morris and Wilson’s ‘systematic approach to change’ model (2004). Key aspects of this model will be expanded upon throughout the following four sections. In cases where examples of activities, discussion posts and messages are given, all names have been changed except that of the author.

Tackling isolation: building a community of learners

It is one thing to facilitate collaborative learning in a traditional, face-to-face classroom setting, but quite another to do so over thousands of miles that span several time zones and cultures. In this context, the learning community exercises some special qualities.

(Irwin and Berge, 2006)

The philosophy of learning communities or ‘communities of practice’ has been promoted by many, although most notably by Lave and Wenger (1991). Through the re-development of these modules we wanted to build a community of learners which would not only act as a support network but also facilitate the sharing of the breadth of knowledge and experience across many industry sectors and national contexts represented by the student body.

For collaborative learning to be successful, learning relationships have to be fostered and an environment of trust needs to be developed and maintained (McConnell, 2002). To support the development of learning relationships within the community the design of the redeveloped modules, each of which incorporated a portal within the institution’s virtual learning environment (VLE), included at least one ‘socialisation’ activity.

Table 1. The challenges of meeting Chickering and Gamson’s Seven Principles in traditional distance education (1987)

Principle	Challenge in traditional distance education
encourages contact between students and faculty	Few opportunities for contact between students and tutors; tutors may be part-time or freelance; students may be unlikely to get an immediate response to queries. Students may perceive that by contacting the tutor they are ‘disturbing’ or ‘annoying’ them; a delayed response may reinforce this perception. Contact between students may be rare or infrequent. Learning activities and assessment tasks are generally designed to be carried out individually.
develops reciprocity and cooperation among students	Limited contact between students precludes co-operative learning. High attrition rates themselves are a barrier to the use of collaborative learning and assessment tasks, as these activities require a critical mass of participation to be of benefit.
encourages active learning	Many distance learning courses provide advice on active study techniques; however, learning in isolation limits the range of activity.
gives prompt feedback	Delays in provision of feedback may be exacerbated by postal submission methods, flexible submission times and off-campus assessors. Little opportunity for peer feedback.
emphasises time on task	Students often have to juggle study with family and work commitments and may find it more difficult than on-campus students to set aside time to focus on their learning. Flexibility in submission deadlines offers both benefits and challenges to the part-time off-campus student.
communicates high expectations	Difficult to determine whether expectations have been communicated successfully. Expectations will be unfamiliar to students who have been out of formal education for several years and may never have studied higher level.
respects diverse talents and ways of learning	earners often restricted to a specific range of learning and assessment methods; e.g. solitary study, writing academic essays. Completion of course dependent on high level of self-motivation, self-organisation and self-discipline.

Salmon (2002) presents three stages to online socialisation:

1. Establishing a successful group.

2. Introducing the knowledge domain and the learning approaches to be used.

3. Introducing how the learning environment is to be used.

In our case the overwhelming majority of students progress through the modules in the same order and their use of the learning environment is fairly similar across the modules. We felt that socialisation activities based on the use of the VLE would be best situated within a separate area to which new students are directed, prior to commencing their studies. We designed an online induction area within the VLE where students are prompted to try out various tools (e.g. forums, shared databases, wikis) alongside other students new to the programme. The individual module areas focus on establishing the group, the knowledge domain and the particular learning approaches for that module.

Figure 1 Approach to change (adapted from Everard, Morris and Wilson, 2004)

We have found that the most successful introductory activities are those that:

1. indicate real-time presence from the tutor. This encourages students to participate as they are keen to make themselves known to the person who will be overseeing their activity and assessing their contributions. A discussion opener that is posted one or two days into the course may be preferable to one that is already set up when students enrol, as this indicates that the tutor is active, online and awaiting their response.

2. introduce the tutor from both a professional and personal perspective and demonstrate the tutor’s empathy with the particular challenges and barriers that students will be coping with as part-time, off-campus students.

3. gives students the opportunity to introduce themselves. However, many students will have already introduced themselves and/or become well-known by the group through previous modules, so it is important to request additional, topical information that they will not have been asked for previously.

An example of an introductory activity that we have used in one of the modules is shown in Figure 2.

Figure 2 Example of introductory ‘socialisation’ activity

Once students have introduced themselves and/or contributed to the introductory task, they are prompted to take part in a number of collaborative learning activities. The exact nature of these varies between modules but in the main they take the form of online discussion. There is widespread agreement in the literature on the value of discussion in interactive online learning environments (Salmon, 2002; Black, 2005; Irwin and Berge, 2006). Effective online discussions can promote reflection and critical thinking through dialogue and enable the participants to collaborate on the construction of meaning (Black, 2005). In our case there was another reason why we were keen to introduce discussion activities with our students: we wanted to link up the static content in the existing learning resources with the fluid ‘content’ that students bring to the modules in the form of their own substantial knowledge and experience.

Design of discussion activities

As Ertmer et al. (2007) point out, ‘neither interaction nor discussion alone is enough to guarantee that students will reach the critical level of learning desired.’ In order to encourage the kind of personal interactions that might lead to critical analysis and higher-order thinking, the discussion tasks we designed required students not merely to post up a response to the original question or task but also to respond to another of their peers’ posts, as highlighted in Figure 3.

This type of activity is similar to those used in Ertmer’s study, where ‘students provided feedback to each other, specifically related to the quality of their postings, with the expectation that this would enable them to grow and learn from each other, and thus, to co-construct knowledge and understanding.’ When selecting questions or topics for discussion, in most cases we adapted the ‘focus questions’ that had originally been presented for individual consideration at the start of each section (each module comprising 6–10 sections). Originally designed for the student to relate the theory covered in that section to their own practice, once adapted for group discussion these tasks enabled students to share and compare perspectives on how the theory related to their own organisations and national contexts. The value of this type of activity in professional terms is of particular significance to engineering, where multinational projects and international placements are commonplace.

Figure 3 Example of group discussion task

If discussion-type activities are felt to be appropriate there are two major issues (beyond the content of the task itself) for the learning designer to consider: participation and moderation.

Participation in any activity relies on a balance of extrinsic and intrinsic motivation. In our case, we chose not to take our chances with intrinsic motivation in the initial stages of the project and made the discussion activities compulsory and credit-bearing, assessed by a tutor according to a simple rubric decided upon by the programme team. However, McConnell (2002) describes an online programme of study where students and staff collaborate on the definition of criteria and participation within the group is managed and assessed by the students themselves. Collaborative learning, as McConnell describes it, is something for all distance educators to aspire to, and the tutor-defined rubric we use to assess student participation in group discussion tasks has limitations. Besides being laborious and time-consuming, tutor assessment of group discussion activities means defining the level of necessary contribution, something which can encourage a mechanistic, detached relationship with the task — a phenomenon we have observed ourselves and a view that McConnell (2002) supports.

Salmon (2002) describes the role of the e-moderator as crucial to the facilitation of successful online discussion. In the redevelopment of our distance learning modules, managing the change in role of the module tutors has been one of the most persistent challenges.

Redeveloping the modules to include online group activities required us to factor moderation of the online activities into the tutor’s contracts. Although some tutors had no difficulty with this new role, others simply could not find the time and back-up moderation had to be provided by the e-learning officer (who was not a subject expert). A varying level of student interaction and engagement between the modules was apparent, and feedback from students clearly highlighted areas where they sensed a need for more input from the tutor. It may be that we placed too much importance on consistency in the format of the learning experience, and this led to us enforcing a rolling schedule of the transfer from self-directed to online collaborative learning that did not allow for differences in the individual tutors’ readiness to participate.

The subtleties of effective online facilitation, particularly when working with students from a variety of cultures and backgrounds, are often complex and, although competency in e-moderation is not difficult to achieve, it does have a vital role to play in developing productive discussion and fostering learning relationships, as demonstrated in Figure 4. In this exchange, a student who is very vocal on the forums is wondering how to respond to another student who has replied to one of his posts privately. The query is framed as a technical one, but the student’s closing comment also gives a sense of social uncertainty as he is implicitly asking for confirmation that these discussions are best held in the open.

In this case the moderator has ‘read between the lines’ and provided a response that is both frank and sensitive to the student’s needs, whilst also answering the student’s original question.

Figure 4 Effective online facilitation of collaborative activity

The availability of subject experts to moderate discussions and respond to individual students will often be limited, depending on whether they are full-time members of staff or working on a part-time or freelance basis. Managing time online is listed as one of Salmon’s core e-moderator competencies (2002), but the workload can be relieved by administrators or dedicated online facilitators who can shoulder some of the burden of activity moderation. However, this is not without its challenges. Problems we have encountered using this system include students not knowing who to contact about a particular issue and subject experts either becoming over-dependent on other members of staff or feeling that their contribution is no longer needed.

A final point on discussion-based activities (which leads on to the next section on assessment): although the activities described above are credit-bearing, these credits currently accumulate towards an additional integrative module and the process is therefore distinct from the assessment of learning outcomes for individual modules. The reasons for this are complex (and primarily bureaucratic) and it is likely that it increases the students’ sense of detachment from the online discussion tasks. The discussions are intended to form the basis of students’ collaborative learning activity during the module but some students only add their contributions after both elements of the module assessment. Our advice to those designing assessment for online modules would therefore be to integrate contribution to the group’s learning into the module assessment pattern where possible.

Enhancing engagement through authentic, collaborative assessment design

Working from a starting point of well-established paper-based distance learning modules has its challenges. Existing practices may be firmly embedded and changing assessment practices can involve more bureaucracy and administrative input than starting from scratch. Although we were keen to effect change across both learning and assessment activities, the persistence of the existing infrastructure restricted us to implementing changes in stages, and within most of the modules an element of compromise between the old and the new remains.

Continuing with our aim of enhancing student engagement we wanted to start moving away from forms of standardised assessment, such as written examinations, that are so distinct from the everyday conditions of learning, teaching and professional practice. Initially, two areas that we chose to focus on in the redevelopment of module assessment were collaborative formative assessment and assessed group role-play.

A full case study of this particular use of group role-play has been published elsewhere by the author (Jordan, 2009), so only a brief summary of the aims and lessons learned follows.

Working from the perspective that the purpose of assessment should incorporate learning itself, and not simply the measurement of learning, our aim was to design a learning and assessment activity that enabled students to work together to develop and demonstrate skills, such as conflict management and dispute resolution, and to apply their understanding of contract law to practice. The activity is based on a fictitious contractual dispute between four parties on a construction project. Groups of students are required to work together in role to resolve the dispute in a process that utilises Web 2.0 tools such as forums and wikis to facilitate asynchronous and synchronous communication and collaborative problem-solving.

The role-play activity is specifically designed to include some of the principles listed previously; particularly the development of reciprocity and co-operation, active learning and prompt feedback between group members. The range of different tasks to be completed by the group throughout the activity allows for diverse talents and ways of learning. The design of the activity also exemplifies the alignment of assessment with the practical application that students are likely to experience in their professional lives.

We have observed a significant increase in student engagement and achievement within this particular module, with student feedback from the module evaluation surveys and conversations with individual students revealing that the ‘clear intentions’ and ‘practicality’ of the role-play activity were key to their increased engagement.

Extensive redesign of assessment practices, as in the above example, takes a significant amount of time and resource, and while developing the role-play activity we also tried to inject an element of authenticity into the other module assignments. The majority of written assignments now focus on the students’ own professional contexts, serving to situate the assessment within the student’s workplace. Students are assessed directly on their ability to apply appropriate theories to their own professional practice. Open-book exams now replace the majority of invigilated ones and also focus on the application of knowledge to practical situations.

Another significant enhancement we have made to the written assignments and open-book exams is to accompany them with collaborative peer- and self-assessment workshops, run on a voluntary basis using the workshop tool within the VLE.

McConnell (2002) writes:

A major objective of collaborative self/peer/tutor assessment is to change the focus of assessment from unilateral tutor assessment (which is usually a summative evaluation, after the event) into a formative learning event. This allows students to learn from the event itself and incorporate what is learned directly into the piece of work they are currently working on.

The workshops allow students not only to benefit from giving and receiving peer feedback but also to gain perspective on how others might solve the same problems. The benefits and challenges we, and the students, experienced from this particular aspect of module redevelopment are presented in the following section.

Enhancing learning through feedback

Ertmer et al. (2007) suggest that online learning is often seen to be feedback-poor, causing students to feel disconnected from the material or the environment, and claim that ‘while instructor feedback is often cited as the catalyst for student learning in online environments, lack of feedback is most often cited as the reason for withdrawing from online courses.’ When we surveyed our students at the start of this project, their response to the question “what can we do to support your learning?” was unanimous: they wanted a lot more formative feedback. Although we expected this response, we felt that a significant increase in provision of tutor feedback was not only unfeasible in terms of resourcing, but also inconsistent with our aim of promoting reciprocity and collaborative learning. This view is supported by Ertmer et al. (2007). The introduction of voluntary, anonymous peer-assessment workshops to some of the modules allowed students to give and receive peer feedback on assignment drafts and answers to sample open-book exam questions.

The value of self and peer assessment in higher education has been widely accepted for several years (Boud, 2000; Broadfoot, 1996). Citing Corgan, Hammer, Margolies and Crossley (2004), Ertmer et al. (2007) present the advantages of peer feedback in online learning environments as: increasing the timeliness of feedback, providing new learning opportunities for both givers and receivers of feedback, humanising the environment and building community. McConnell (2002) also reminds us that student involvement in their own assessment is ‘an important part of the preparation for life and work’.

The fact that our peer assessment workshops were voluntary and anonymous precluded any significant contribution to community building and we would suggest that others bear in mind that opting for anonymity and participatory choice may compromise the level of reciprocity and collaboration achieved. While McConnell (2002) believes that ‘the openness of the collaborative assessment process is crucial to its success’, some (McConnell cites Ames (1992) as an example) believe that all feedback should remain private. Wherever one stands within this particular debate, as an initial venture into peer assessment voluntary and anonymous activities involve little risk.

Although no direct evidence of community building was expected from the anonymous peer assessment workshops, participating students informed us (through the module evaluation surveys) that they appreciated the timely nature of the feedback and its value to their learning, the opportunity to actively engage with the assessment criteria and the chance to get better acquainted with their peers’ experiences and perspectives.

Liu et al. (2001) suggest that, beyond the obvious benefits to students of greater familiarity with the assessment criteria and learning from the content of their peers’ assignments, giving peer feedback on a task can utilise higher cognitive processes than the task itself. In assessing their peers’ work, students have to ‘read, compare, or question ideas, suggest modifications, or even reflect on how well one’s own work is compared with others’. McConnell (2002) adds that by engaging with peer assessment students are developing skills that they can use to assess their own learning and, citing Boud (2000), ‘equipping learners with such skills should be a key aspect of the so-called learning society’.

In addition to increasing the amount and variety of feedback that students receive through peer assessment workshops, we also wanted to find a means of ensuring that they will engage effectively with summative feedback received from the tutor and reflect on how they will take that feedback forward into future study. We did this by adding an additional step to the summative feedback process for every written assignment where students were asked to respond to their assignment feedback, as shown in Figure 5.

This activity prompts students to focus on the formative aspects of their feedback and to identify where they might be able to apply it in the future. It also gives students an opportunity to engage in dialogue with the tutor in order to clarify any points that may have been poorly understood.

Figure 5 Example of feedback response task

The responses received by the students varied greatly in terms of depth and perspective on the task. A small number of students, as we expected, took this activity as an opportunity to question or challenge the tutor’s assessment of their work. We take the view that if a student feels that the assessment of their work has been unfair, it is better that they tell us about it so that we can attempt to resolve the disagreement and identify how we can reduce the likelihood of it happening in the future.

Many students demonstrated impressive application to the task and it was clear from their response to the activity that they felt they had benefited educationally from engaging with their feedback in such an explicit way. Feedback regarding this particular aspect of module redevelopment (whether taken directly from the students’ responses, the module evaluation surveys or private communication with individual students) was overwhelmingly positive.

Developing a learning community of educators

Every institution will face different barriers to pedagogical change. For us the fact that our module tutors are freelance and off-campus was the most significant factor in the challenges we experienced.

Although tutors took an active part in considering the direction of module redevelopment, the degree to which they wanted to be involved in the design of the redeveloped modules was variable. Lack of familiarity with the VLE appeared to cause problems for some tutors and not others: some (despite never having used the VLE previously) took to it immediately, while others continued to report feelings of confusion and incomprehension even after a number of online and face-to-face orientation sessions.

As the schedule of module redevelopment progressed, a decision was taken to apply similar techniques used to build effective student learning communities to the development of a tutor community. The aim of this was to support the change process by building supportive relationships between the module tutors and encouraging the sharing of good practice. A community area was set up within the VLE to which all the tutors were invited to share their ideas on, for example, what constitutes a good learning experience, and given space to freely air views on the challenges and benefits of change.

The community area saw a lower amount of active participation then the programme management team hoped for, although it was apparent that many tutors had accessed the area and it was successful in raising awareness of practice in other modules. One discussion led to a determined outburst from one tutor on matters largely unrelated to pedagogical innovation, but this at least triggered further discussion. It also gave us the opportunity to respond constructively and transparently to the tutor’s concerns. With hindsight, we realised that facilitating relationships between module tutors and building a community of practice prior to module redevelopment may have helped to overcome some of the challenges we experienced in effecting pedagogical change.

Conclusions

The following key points may be useful for others to consider when designing for collaborative distance learning.

Introductory socialisation activities are important for building effective relationships between the members of the learning community prior to collaborative learning activities such as discussion tasks.

Discussion activities can help to promote collaborative learning, but careful consideration needs to be given to the means of encouraging student participation. Activities where participation is enforced through tutor assessment of student contributions may promote a mechanistic response to the task and/or create a power differential that may hinder the educational value of the activity. There are ways in which this power differential can be reduced (for example if students peer assess contributions and/or collaborate on the definition of criteria) and these should be considered when designing the activity. If contribution to the group’s learning is to be assessed then it should be integrated as seamlessly as possible into the module assessment pattern.

Moderation of online discussions is time consuming and it is worth considering whether it is appropriate for other members of staff to take some of the moderation workload from the shoulders of the subject experts. If this strategy is used it is important to ensure that both students and staff have a clear understanding of different staff roles and that a general forum is used for students to raise non-personal queries. This allows the query to be answered by the most appropriate member of staff and should also preclude duplication of effort.

Module tutors should be involved as much as possible in diagnostic and planning work and flexibility should be utilised when implementing a schedule of new developments. If it transpires that a tutor isn’t quite ready to commit to the facilitation of online group work then it may be better for them to shadow another tutor at first, rather than risk providing students with an initial online learning experience that does not meet their expectations.

Assessed activities can have a significant, positive impact on engagement if they are social and collaborative and if their authenticity allows students to see the relevance of the theory and apply it in a practical way. The use of role-play has particular benefits in cases where students need to demonstrate inventiveness and creative problem-solving.

Peer and self assessment have long been championed as a means of increasing engagement and developing students’ ability to assess their own learning. Open (as opposed to anonymous and private) peer assessment can be an extremely worthwhile educational experience but requires an environment of reciprocity and mutual trust (McConnell, 2002).

Asking students to respond to their feedback has been found to be a highly effective method of increasing distance learners’ engagement with summative feedback. Feedback response tasks encourage students to focus on how they will use their feedback in the future, while simultaneously allowing them to enter into a dialogue with the tutor on any points that require clarification.

Our advice to teams attempting to engineer a similar programme of redevelopment with existing members of staff would be to engage in staff community development activity prior to attempting to change pedagogical practices. Hord’s (1997) dimensions of a professional learning community (cited in Hipp, 2008) include shared leadership, shared values and vision and collective learning and application, all of which are important in the pursuit of sustainable pedagogical change (Owston, 2007).