Do we always need a difference? Testing equivalence in a blended learning setting

ABSTRACT

Evidence-based research is becoming increasingly important in educational research. Calculation and test methods available in statistical software packages such as SPSS and STATA are widely used. To evaluate teaching innovations such as blended learning against classical classroom settings, for example, previous studies have mainly applied inference methods such as the t-test or variance analyses. The problem with these methods is that they test for the difference. A non-significant result does not automatically mean equivalence of the treatments examined, which is why we propose the use of equivalence testing. This paper introduces the equivalence test as complementary to the classical t-test and briefly discusses other approaches based on confidence intervals and Bayesian methods. As an example, the introduction of a blended learning format to a Bachelor's degree programme is used to demonstrate the procedure and discuss the results of conducting an equivalence test. By combining tests for difference and equivalence successfully, it was possible to arrive at more informative statistical statements: Whereas a t-test alone only produced results for three out of 22 courses, a t-test and an equivalence test in combination yielded statistically confirmed statements for 12 out of 22 courses.

KEYWORDS:

• Blended learning
• equivalence testing
• flexible learning
• learning effectiveness
• methodology

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Claude Mueller http://orcid.org/0000-0003-1987-6662

Notes

1 See detailed references in Vo, Zhu, and Diep (2017).