Development and implementation of an online hybrid model for teaching evidence-based practice to health professions: processes and outcomes from an Australian experience

Abstract

Evidence-based practice is now considered to be a vital element of health care service delivery. The call to use evidence to inform other areas, such as teaching and learning, is growing. This paper reports on the processes used to integrate best evidence into teaching practices within an undergraduate health science program. An existing course within this program at an Australian tertiary institution was remodeled by a newly appointed course coordinator in response to critical feedback from student cohorts. A systematic, iterative, five-step approach was used in the development of the new course. The process of development was influenced by current research evidence, an audit of the existing course, and critical feedback from students. The new course was evaluated using quantitative and qualitative research methods for five study periods. In 2005, prior to implementing the changes, the overall student satisfaction rating for the course was zero (representing the lowest possible score). In 2006, the overall student satisfaction rating was 62.07, in 2007 it was 65.8, and in 2008 it was 55.7. Qualitative findings also supported these quantitative findings, indicating improvements in the structure and process of the new course. The outcomes from the evaluation of the remodeled course provide evidence of a consistent quality learning experience for students, and support the concept of using research evidence to guide the development of teaching and learning practices in the training of health professionals.

Keywords:

• evidence-based teaching
• learning
• health care
• qualitative
• quantitative

Introduction

Health care service delivery has been transformed in recent years. There is increasing recognition that health care practices, which have traditionally relied upon historic and experiential evidence, now need to be underpinned by rigorous research evidence.¹ The reasons for this change are numerous and include an increasing burden on health care systems from chronic diseases and their associated conditions, an aging population, diminishing health care resources due to competing funding interests, and persistent shortages of health care professionals.^2–4 These drivers have identified the need to ensure scare health care resources are efficiently utilized. Evidence-based practice (EBP) is now considered as an important vehicle in improving the quality and safety of health care practices and ensuring optimal patient outcomes.⁵

The emergence of EBP has also profoundly influenced teaching practices for students of various health professions. Many tertiary educational institutions now routinely use EBP principles within their curricula, either as stand-alone courses or embedded into the existing curricula.⁶ EBP principles underpin the importance of using research evidence to inform practice, recognize local context and clinical expertise, and respect patients’ values and beliefs. The outcome of this is that students of health disciplines are exposed to theoretic and practical aspects of EBP and its relevance to current and future health care practices. The outcome of integrating EBP principles as part of a teaching program is the creation of a contemporary health care practitioner, who embraces lifelong learning principles, possesses skills in accessing and evaluating research, and who recognizes the integral role of patient-centered health care.

In recent times, the notion of using research evidence to inform health care practices has extended into teaching and learning domains.^7,8 Stakeholders in education are increasingly demanding that teaching and learning processes be underpinned by research evidence. Utilizing research evidence to inform teaching practices, which borrows on the principles of EBP, has been termed evidence-based teaching.⁷ The concept of linking research and teaching to improve teaching outcomes has also been termed “the teaching-research nexus”.⁹ The teaching-research nexus recognizes the relationship between teaching and research as two academic activities that are mutually beneficial. Irrespective of how this concept is termed, the twenty-first century has introduced numerous novel models of teaching and learning. An example of this includes the increasing use of online education.

Online education embraces novel delivery modes which can include audio, video, and computer systems.¹⁰ A randomized controlled trial demonstrated that a computer-based teaching package was as effective as a lecture-based teaching session for teaching EBP principles to doctors.¹¹ There is growing evidence that online education is being slowly integrated into many tertiary educational institutions.^12,13 The drivers for this change have been numerous, and include improved time and resource efficiencies from a teaching perspective and improved access to course material for students. Online education can complement the strengths of current student cohorts, including their enhanced understanding and knowledge of current information technologies.¹⁴ Evidence-based practice can be readily applied to online education models because it combines theoretic concepts and practical applications into simple, standalone modules. Furthermore, with EBP gaining momentum, there are numerous freely available online resources which could be utilized by students.

Implementing change

Call for action

An opportunity arose to use research evidence to inform teaching practices at an Australian tertiary institution. An existing online course on EBP, which had been introduced one year prior, was remodeled by a newly appointed course coordinator, in response to critical feedback from student cohorts. Routine evaluation of teaching at the educational institution identified several opportunities for improvement. Quantitative evaluation (undertaken using standard course evaluation instruments) identified poor ratings of students’ satisfaction with the course. Qualitative evaluation (undertaken using focus groups of students) identified numerous issues. These included structural problems (lack of access to the course coordinator, lack of adequate background information about the course, and poor organization of web pages), course content (course being “too easy” with much repetition and ambiguity, minimal examples), process issues (poor and untimely feedback from educator, lack of availability of educator, irregular communication between educator and students, lack of guidance and support in successfully completing assessments), and poor value for money. Based on these findings, it was clear that the current EBP course needed to be revamped.

Processes of change

A systematic approach was considered before making large-scale changes to this course. It was decided that an iterative, step-by-step approach would be undertaken, informed by evidence from the literature. In consultation with key stakeholders within the institution, a five-step approach was initiated to strengthen the structure, content, and processes of the course. Step One was to establish a baseline in terms of structure, process, and content. This was achieved through an audit of the existing course (supported course review offered at the educational institution). Step Two was to identify current industry standards in teaching and delivery of similar courses. This was achieved by communication with educators at other tertiary institutions. Step Three was to identify research evidence on best practice for online teaching and learning. This was achieved through a review of the literature. Step Four was a staged implementation plan, incorporating lessons learnt from Steps Two and Three. Gradual changes to the course were undertaken over a period of two semesters (also called study periods). Step Five was to embed regular monitoring strategies into the course, undertaken via qualitative and quantitative surveys. This step was intended to help capture students’ perspectives on the new course and ensure that key learning outcomes were achieved. This five-step approach is described in Figure 1. No additional funding, other than the appointment of the new course coordinator, was provided to facilitate this process of change.

Figure 1 The five step process of change.

A hybrid online model

Key findings from Step One and Step Two indicated the need for a model of online teaching and learning to suit the needs of all relevant stakeholders. From the students’ perspectives, although the course was delivered online, it was important that the educator still had a visible presence and was readily accessible. Investigation of other courses (Step Two) revealed that a mixed model of education provision was often used, whereby some components of courses involved face-to-face contact between the educator and the students, and some were conducted using online learning resources. Recognizing these requirements and other local drivers, a review of the literature on best practice for online teaching and learning was conducted, and a large volume of literature was subsequently identified and retrieved. Evaluation of relevant literature highlighted the existence of a hybrid online model.^10,15–20 This model of online education has been termed “blended learning”. According to Martyn, blended learning provides opportunities to use multiple technology modes, combines pedagogic approaches, and may include blending of educational technologies with face-to-face student and educator interaction.¹⁰ Based on the broad principles of this hybrid/blended online model, and taking into account local context and drivers, such as the information technology tools available, and educational and administrative requirements, a model specific to this course was developed. This model is displayed in Figure 2.

Figure 2 UniSA hybrid online model (adapted from Martyn 2003¹⁰).

Key constructs underpinning the hybrid online model

The model presented in Figure 2 was underpinned by student-centered learning principles, whereby collaborative decisions were made between an educator and the student cohort on key aspects of course delivery. This process empowered students by residing ownership of course content and delivery with the student collective. These constructs are derived from the processes described by Martyn.¹⁰ The key constructs embedded within this model are presented and briefly discussed below.

Face-to-face lectures

A series of face-to face-lectures were programmed throughout the study period. The first lecture was scheduled during the initial week of the study period during which students interacted with the educator.

This face-to-face meeting provided the ideal opportunity for students to meet with the educator, “put a face to a name”, and to discuss the objectives of the course. For many students, this was the first course which was delivered predominantly online. Therefore, the educator expressly acknowledged the notion that online learning may be new to many students and not particularly easy, especially in the initial stages. During this lecture, students were also introduced to the course webpage and online learning resources in a step-by-step manner. By doing this, students were not confronted with an entirely new webpage when they logged on. The online learning resources contained content materials, discussion pages, links to resources, and assessment details.

A subsequent face-to-face lecture introduced students to the concepts of the course and provided an overview of important aspects of the course that would be covered throughout the study period. These lectures aimed to familiarize students with concepts of the course using a form of teaching they were familiar with (face-to-face). At the midpoint of the study period, another face-to-face lecture was included, with the aim of providing students with feedback from their first assignment, and providing opportunities to resolve any problems or queries. A final face-to-face lecture was scheduled at the end of the study period to provide closure for the course. Students were encouraged to provide feedback on the course and discuss strategies for improvement. This meeting gave rise to productive discussions on course content, delivery, and outcomes. One such suggestion was to include podcasts of lectures, which was subsequently implemented.

Emails and educator feedback

The educator and students routinely communicated by email. Because access to the educator was identified as an integral issue in the evaluation stage (audit and focus groups), a 24-hour response time was set as a benchmark for working hours and weekdays throughout the study period. This benchmark ensured timely responses to student queries. Students had the opportunity to either email the educator directly or address the whole class via the email distribution list. This ensured that students had access not only to the educator but also to their peers. An additional software tool, Assign IT, was utilized to ensure timely and adequate feedback for assignments. Assignments along with feedback were returned electronically by using track changes, a function of Microsoft Word. By using these tools, the educator could assess assignments and immediately return marked assignments with comments. These processes ensured that students did not have to wait long to obtain feedback, enabling them to incorporate feedback immediately into subsequent assignments.

Asynchronous online discussions

The asynchronous online discussion boards were primarily tools of learning for this course. These discussions provided students with the opportunity to post and discuss issues relevant to aspects of the course with their educator and among their peers. Some online discussions were primarily focused on set tasks which were directly related to principles underpinning the course. Other discussions related to a general introduction (as students were a mix of undergraduate and postgraduate physiotherapy students), students’ general perspective of the course, and issues specific to individual assignments. A “Frequently Asked Questions” discussion board was created close to each assignment due date, where previous cohorts’ questions and answers were posted. This provided opportunities for students to reflect on previous discussions and post new discussions or new queries. Online discussions specific to aspects of the course attracted grades, which contributed towards the overall assessment. Students contributing to indepth discussions and encouraging good quality discussions earned higher grades.

Evaluation and student feedback

The revamped course was evaluated at the end of five study periods between 2006 and 2008. The anonymous course evaluation was conducted online as part of educational institutions’ Course Evaluation Instrument (CEI) and Student Experience of Teaching (SET). These tools are shown in Figures 3 and 4. These evaluation tools are routinely used for each course offered at the educational institution. This allows data to be compared within courses and across courses at the educational institution. Quantitative evaluation using CEI from 2006, 2007, and 2008 indicated improvements in students’ satisfaction with the course. In 2005, prior to implementing the hybrid model, the mean score for question 10 in the CEI (“Overall I was satisfied with the quality of this course”) was 0. No other evaluation data for 2005 were made available to the new course coordinator because of ethical and confidentiality reasons. In 2006 the mean score for this question was 62.07, in 2007 the mean score was 65.8, and in 2008 the mean score was 55.7 (data from only one study period was available in 2008). Quantitative evaluation using SET from 2006, 2007, and 2008 also indicated improvements in students’ satisfaction with the educator for this course. In 2006, the mean score for question 10 in the SET (“Overall, I was satisfied with the performance of this staff member”) was 82.7, in 2007 the mean score was 86.2, and in 2008 the mean score was 86.5 (data from only one study period was available in 2008).

Figure 3 Course evaluation instrument.This evaluation form may consist of two types of questions. The first type asks you to respond to a series of statements by indicating your agreement or disagreement with each of the statements. It is important that you respond to each item. The second type is a straightforward question to which you respond with text. You do not have to complete the text response items.

Abbreviations: SA, strongly agree; A, agree; N, neutral; D, disagree; SD, strongly disagree.

Figure 4 Student evaluation instrument. This evaluation form may consist of two types of questions. The first type asks you to respond to a series of statements by indicating your agreement or disagreement with each of the statements. It is important that you respond to each item. The second type is a straightforward question to which you respond with text. You do not have to complete the text response items.

Abbreviations: SA, strongly agree; A, agree; N, neutral; D, disagree; SD, strongly disagree.

Qualitative evaluation using CEI and SET also revealed positive findings. Students commented positively on the structure of the revamped course (“Well constructed and supported”, “The course structure was flexible. It allowed me to pursue independent learning”, “Good amount of lectures – enough to give information required for learning and assignments but not too many to bore”), the content (“The work load was well proportioned with respect to other subjects”, “…once you begin to understand the content of the course you realize how important it is”), communication between educator and students (“Principal tutor was very patient and knowledgeable!”, “Staff easy to approach”, “Enthusiastic. He made the lectures as interesting as he could, and was always able to respond to questions and emails quickly”), competencies of educator (“His availability to answer questions and provide assistance, his comments in the discussion forums”, “Lecturer made course content interesting”), and improved opportunities for feedback (“The course coordinator provided great feedback on discussion forums, very approachable, always willing to help, willing to alter due dates of assignments, use of up-to-date teaching methods”).

Discussion

The process and outcome of the remodeling of an existing course presented in this paper illustrates several important issues which relate to the integration of research evidence into teaching in a tertiary educational institution. Firstly, despite the growing body of evidence supporting the use of online teaching and learning, it appears that maintaining face-to-face contact with an educator throughout a course may be an essential ingredient for its success. This seems especially true in the early stages of a course because students, while proficient in the use of electronic media, seem to require direction and assistance in transferring these skills, gained mostly from social interactions, for educational purposes. Secondly, online teaching and learning, while providing flexibilities and efficiencies, runs the risk of students being “out of sight, out of mind”. Therefore regular monitoring of the progress of students and regular feedback from educators seem to be vital to ensure student satisfaction and positive learning outcomes. Use of multiple resources, such as face-to-face contact, online discussions, and podcasting, appears to be favorably received by students because it provides a wide range of tools to access learning materials.

This paper highlights the importance of using systematic processes to integrate research evidence into teaching practice. The outcomes achieved as a result of this systematic process provide confirmation that integrating evidence from different sources, such as research and stakeholders, is practical and achievable. The outcomes also highlight that while online teaching and learning media are highly useful, students still require face-to-face instruction and timely feedback. Relying on technology alone, without complementing it with human interface, may not lead to successful learning outcomes.

While there were positive findings from this project, it is important to recognize a number of limitations inherent in its structures and processes. First, this project was spearheaded by one course coordinator and, therefore, it is possible that the positive outcomes attained were directly as a result of that educator’s involvement (rather than changes to the course processes). This will limit generalizability and applicability of findings to the wider community. However, it was difficult to “unpick” the role of the educator from the changes to the course processes, as these two are intrinsically linked. Second, as this course was developed and delivered within an Australian tertiary institution, certain elements of the course had to conform to the institution’s routine procedures. As the CEI and SET were standard tools used to garner students’ perspectives of the course and teaching (customary evaluation tools), no modifications could be made. Furthermore, completion of these survey tools was entirely voluntary. Therefore, data collected using these tools are of limited value due to their generic nature, lack of psychometric testing, voluntary participation of students, and focus on students’ satisfaction (rather than learning). Finally, undertaking evaluations for such interventions provided in such complex, uncontrolled settings, where students’ perspectives of a course and teaching can be influenced by a myriad of factors, means that the findings should be interpreted with caution.

Implications for teaching and learning

The use of research evidence to inform and evaluate practice in teaching and learning domains poses many challenges for educators and students alike. While online education may improve access and convenience, it can also be a daunting experience for novice educators and students. It is clear from this project that the processes used to remodel this course were effective in improving the quality and outcomes of teaching practices for this course. While recognizing its limitations, the processes described in this paper could provide a practical framework which can be transferred to teaching and learning practices in other contexts.

Implications for research

Systematic evaluations of teaching and learning practices for health professions are scant in the literature. While there is a growing body of research evidence on online education, systematic evaluation and monitoring of other models of teaching and learning are required. Such evaluations will help build an evidence base for various models of teaching and learning for health care professionals. This paper demonstrates how multiple educational strategies (face-to-face lectures, online discussions, podcasts) can be more effective than online education alone. Further research is required to determine the effectiveness of multiple strategies (when compared with single strategies) and optimal educational packages.

Conclusion

As technology continues to evolve, different models of teaching and learning (such as online education) will increasingly be utilized by many educational institutions as a means of improving efficiencies, access, and flexibility. Despite its widespread use, online education continues to pose many challenges for educators and students alike. New models of teaching and learning may prove to be more effective if they are constructed systematically, and underpinned by evidence from research within these fields. This paper highlights the benefits of using such an approach, while acknowledging limitations and barriers in undertaking research in these settings. The processes reported in this paper may provide a template for educators or educational institutions wanting to embed research evidence into teaching and learning and reap the rewards.

Disclosure

The authors report no conflicts of interest in this work.