Online and paper evaluations of courses: a literature review and case study

Abstract

This paper reviews the literature on comparing online and paper course evaluations in higher education and provides a case study of a very large randomised trial on the topic. It presents a mixed but generally optimistic picture of online course evaluations with respect to response rates, what they indicate, and how to increase them. The paper presents a case study of 1 university and finds that means for paper course evaluations tend to be higher than for online evaluations, that the standard deviations of online evaluations are typically larger than for paper evaluations, that online evaluations take longer to complete than their paper counterparts, that students prefer online evaluations, and that factor analysis shows similar and different numbers of factors for the 2 types of evaluation, even with the same instrument and the same population. Caution is advocated in assuming that the same online and paper evaluations yield similar results.

Keywords:

• online evaluation
• course evaluation
• SET
• paper evaluation
• quality assurance

Notes

1. H. M. Anderson, Cain, and Bird (2005); Donovan, Mader, and Shinsky (2007); Fike, Doyle, and Connelly (2010); Johnson (2003); Layne, DeCristoforo, and McGinty (1999); Stowell, Addison, and Smith (2012).

2. Baum, Chapman, Dommeyer, and Hanna (2001); Collings and Ballantyne (2004); Crews and Curtis (2011); Dommeyer, Baum, Chapman, and Hanna (2002); Dommeyer, Baum, Hanna, and Chapman (2004); Donovan, Mader, and Shinsky (2006, 2007); Fike et al. (2010); Guder and Malliaris (2010); Ha, Marsh, and Jones (1998); Hardy (2003); Hmieleski and Champagne (2000); Johnson (2003); Kasiar, Schroeder, and Holstad (2002); Kuhtman (2004); Layne et al. (1999); Ravelli (2000); Rhea, Rovai, Ponton, Derrick, and Davis (2007); Sorenson and Johnson (2003); Sorenson and Reiner (2003); Stowell et al. (2012); Tucker, Jones, Straker, and Cole (2003).

3. H. M. Anderson et al. (2005); Bothell and Henderson (2003); Crews and Curtis (2011); Cummings, Ballantyne, and Fowler (2000); Dommeyer et al. (2002); Dommeyer et al. (2004); Donovan et al. (2006, 2007); Hmieleski and Champagne (2000); Johnson (2003); Kuhtman (2004); Layne et al. (1999); McCracken and Kelly (2011); Nulty (2008); Sorenson and Reiner (2003); Stowell et al. (2012).

4. Ballantyne (2003); Bothell and Henderson (2003); Crews and Curtis (2011); Fraze, Hardin, Brashears, Smith, and Lockaby (2000); Hmieleski and Champagne (2000); Johnson (2003); Kronholm et al. (1999); McCracken and Kelly (2011); Sorenson and Reiner (2003).

5. H. M. Anderson et al. (2005); Badri, Abdulla, Kamali, and Dodeen (2006); Barkhi and Williams (2010); Crews and Curtis (2011); Dommeyer et al. (2004); Donovan et al. (2007).

6. H. M. Anderson et al. (2005); J. Anderson, Brown, and Spaeth (2006); Crews and Curtis (2011); Donovan et al. (2007); Hmieleski and Champagne (2000); McCracken and Kelly (2011); Watt, Simpson, McKillop, and Nunn (2002).

7. Avery, Bryant, Mathios, Kang, and Bell (2006); Chang (2003); Crews and Curtis (2011); Dommeyer et al. (2002); Dommeyer et al. (2004); Guder and Malliaris (2010); Laubsch (2006); Layne et al. (1999); Liegle and McDonald (2005); Norris and Conn (2005); Richardson (2005); Schawitch (2005); Sorenson and Johnson (2003); Sorenson and Reiner (2003); Stowell et al. (2012); Thorpe (2002).

8. Avery et al. (2006); Barkhi and Williams (2010); Carini et al. (2003); Dommeyer et al. (2002); Dommeyer et al. (2004); Donovan et al. (2006); Fike et al. (2010); Gamliel and Davidovitz (2005); Guder and Malliaris (2010); Ha et al. (1998); Handwerk, Carson, and Blackwell (200l); Heath, Lawyer, and Rasmussen (2007); Johnson, 2003; Layne et al. (1999); Perrett (2013); Sorenson and Johnson (2003); Stowell et al. (2012); Thorpe (2002); Turhan, Yaris, and Nural (2005).