Balancing the effectiveness and cost of online education: A preliminary realist economic evaluation

Abstract

Purpose

While online learning for faculty development has grown substantially over recent decades, it has been further accelerated in the face of the worldwide pandemic. The effectiveness of online learning has been repeatedly established through systematic reviews and meta-analyses, yet questions remain about its cost-effectiveness. This study evaluates how synchronous online supervision training workshops and their cost-effectiveness might work, and in what contexts.

Methods

We conducted preliminary realist economic evaluation including qualitative (13 realist interviews), and quantitative approaches (cost Ingredients method). We developed a cost-optimised model based on identified costs and cost-sensitive mechanisms.

Results

We identified 14 recurring patterns (so-called demi-regularities) illustrating multiple online workshop outcomes (e.g. satisfaction, engagement, knowledge), generated by various mechanisms (e.g. online technology, mixed pedagogies involving didactic and active/experiential learning, peer learning), and triggered by two contexts (supervisor experience levels, and workplace location). Each workshop cost $302.92 per learner, but the optimised model including senior facilitators cost $305.70.

Conclusions

Our initial realist program theories were largely supported and refined. Although findings were largely concordant with previous literature, we illustrate how online workshop costs compare favourably with face-to-face alternatives. We encourage program developers to consider synchronous online learning for faculty development especially for remote learners, and in resource-constrained environments.

Keywords:

• Supervision training
• online learning
• realist evaluation
• realist economic evaluation

Acknowledgements

We thank our colleagues involved in the broader realist evaluation of the supervision training program (Corinne Davis, Sarah Lee, Ella Ottrey, Kirsty Pope, and Susan Waller).

Ethical approval

We received ethics approval from the Monash University Human Research Ethics Committee (Project 13151 and 19269).

Authors contributions

CER and VE designed and secured funding for this supervision training and evaluation program. VNBN collected realist evaluation data for online workshops, while JF collected economic data. All authors contributed to the analysis of the realist economic evaluation. CER and CP are both Principal Investigators. CER wrote the paper, with VNBN and JF contributing to the realist and economic aspects of the methods and results, respectively. The three remaining authors edited the final iteration of the paper. All authors give their final approval for this version to be published.

Disclosure statement

Two authors were involved in program design (CER, VE), three led program management (CER, VE, and CP), and one contributed to the delivery of online workshops (CP). However, the remaining two authors (JF, SM) were not involved in the workshop design or delivery, so have no conflicts of interest.

Additional information

Funding

The Victorian Government funded this supervision training and evaluation program.

Notes on contributors

Charlotte E. Rees

Charlotte E. Rees, BSc (Hons), GradCertTerEd(Mgt), MEd, PhD, Head of School, School of Health Sciences, College of Health, Medicine and Wellbeing, The University of Newcastle, Callaghan, NSW, Australia. Adjunct Professor, Monash Centre for Scholarship in Health Education (MCSHE), Faculty of Medicine, Nursing & Health Sciences, Monash University, Clayton, VIC, Australia.

Van N. B. Nguyen

Van N. B. Nguyen, BN, MN, PhD, Research and Teaching Associate, Monash Nursing & Midwifery, Faculty of Medicine, Nursing & Health Sciences, Monash University, Clayton, VIC, Australia.

Jonathan Foo

Jonathan Foo, BPhysio (Hons), PhD, Lecturer, Department of Physiotherapy, School of Primary and Allied Health Care, Faculty of Medicine, Nursing & Health Sciences, Monash University, Frankston, VIC, Australia.

Vicki Edouard

Vicki Edouard, BSc (Hons), GradDipEdu, Department Manager, Department of Mechanical and Aerospace Engineering, Monash University, Clayton, VIC, Australia.

Stephen Maloney

Stephen Maloney, BPhysio, MPH, eMBA, PhD, Deputy Head of School, Department of Physiotherapy, School of Primary and Allied Health Care, Faculty of Medicine, Nursing & Health Sciences, Monash University, Frankston, VIC, Australia.

Claire Palermo

Claire Palermo, BSc, GradCertHlthProfEd, MNutDiet, MPH, PhD, Director, Monash Centre for Scholarship in Health Education (MCSHE), Faculty of Medicine, Nursing & Health Sciences, Monash University, Clayton, VIC, Australia. Assocate Dean (Teaching and Learning), Faculty of Medicine, Nursing & Health Sciences, Monash University, Clayton, VIC, Australia.