https://orcid.org/0000-0002-4276-3976
ABSTRACT
The COVID-19 lockdown event in early 2020 provided a rare opportunity to directly compare students’ experience in the remote online classroom with their prior experience in the traditional physical classroom. A total of 354 survey responses were analysed statistically. Students’ experience in the remote online classroom was found to be less satisfactory than their experience in the traditional physical classroom. Interaction with fellow students was not satisfactory in the remote online classroom. Interactions with faculty, other students, class materials and educational technology were negatively influenced by the move to remote online learning. Home environment correlated with student learning experience in the remote online classroom but not in the physical classroom. The ambience in the home – noise, disturbance and distraction – was only marginally satisfactory for undergraduate students. Master students were more satisfied with their online classroom experience and with internet connectivity and ambience in their home than were undergraduates.
Introduction
This paper examines third level students’ satisfaction with their experience of remote online learning that was brought on unexpectedly by the COVID-19 pandemic in early 2020. The global lockdown response resulted in an estimated one and a half billion learners suddenly undertaking lessons and courses remotely and academics working and teaching from home (Bozkurt & Sharma, Citation2020; Coates et al., Citation2021; Moja, Citation2021). For many countries, including the author’s own country Ireland, the lockdown began in March 2020 during the Spring semester. This meant that teaching and learning in the early part of the semester took place in the traditional, physical, classroom and in the later part of the semester in an online, virtual, classroom with students and lecturers working from their homes i.e. remotely from the university. This event provided a unique opportunity to directly compare students’ experience of remote online learning with their earlier experience of the same course given by the same lecturer in the physical classroom.
To explore the impact of the pandemic event on student learning the paper draws on two theories: interaction theory and social cognitive theory. Interaction is a critical part of the educational process both online and on campus (Anderson, Citation2003); it is fundamental to the connectivist approach to education with its emphasis on wayfinding and sensemaking, operations and innovation (Wang et al., Citation2014). Collaborative modes of learning and opportunities for discussion have been found to enhance student learning (Ertl & Wright, Citation2008). Interaction has been found to influence student satisfaction and performance (Kuo & Belland, Citation2016) whereas lack of interaction with instructors and fellow students, and difficulties with technology, diminished learning (Elshami et al., Citation2020). Moore (Citation1989) put forward three distinct modes of interaction that are relevant to student learning: interaction between learner and content, between learner and instructor, and between learner and other learners. The interaction model was subsequently expanded to include technology (Danesh et al., Citation2015).
Social cognitive theory suggests that effective learning requires more than conditioning, reward and punishment: it emphasises that humans learn from each other through observation, imitation and modelling (Bandura, Citation2001). Humans are active agents in the learning process and not simply passive learners; they are proactive, self-organising, self-reflecting and self-regulating. People continually interact with their environment and the triad of personal, behavioural and environmental factors act as determinants of human learning (Bandura, Citation2001). The theoretical underpinning of this paper – the environment, educational interactions and the student experience – is depicted graphically in .
Figure 1. Theoretical underpinning.
The pandemic in Spring 2020 resulted in a major change in the environment of students. Students and faculty were abruptly sent away from the university to work from their homes. Classes however continued, albeit in a new remote online format. Universities as professional teaching institutes, and faculty as professional instructors, endeavoured to provide a satisfactory learning environment for students despite the changed and difficult circumstances. Interactions between faculty, fellow students and course materials took place from faculty and students’ homes via the new online virtual environment. The mechanisms available to students to observe, imitate and model themselves on faculty and other students altered radically. Although modern communications technologies provide increased opportunities for learning (Bandura, Citation2006), it is likely that, given the unplanned and unexpected nature of the move, the learning experience in the remote online classroom was less satisfactory for students than was their prior experience in the physical classroom. This line of argument leads to the following group of hypotheses:
H1a:
Students were more satisfied with their learning experience in the physical classroom than with their learning experience in the remote online classroom.
H1b (c; d; e):
Students were more satisfied with rapport with the lecturer (interaction with fellow students; engagement with class materials; educational technology) in the physical classroom than in the remote online classroom.
Students worked from home during the second half of Spring 2020, and it is likely that students’ home environment impacted on their learning with better home situations providing a better learning experience. Given that masters students are more experienced than undergraduate students, both academically in that they have already completed their undergraduate education, and in terms of life experience in that they are usually older than undergraduates, it is likely that their home situations were more satisfactory for learning than were those of undergraduate students. Due to their better home situations, it is likely that master level students had a more satisfactory remote learning experience during the pandemic than did undergraduate level students, leading to the second group of hypotheses:
H2a (b; c; d).
Satisfaction with students’ learning experience in the online classroom is positively related to satisfaction with students’ home internet connection (furnitureFootnote1; ambienceFootnote2; environmentFootnote3).
H3a (b; c; d):
Satisfaction with students’ home internet connection (furniture; ambience; environment) is greater for master level students than for undergraduate level students.
H3e.
Satisfaction with students’ learning experience in the online classroom is greater for master level students than for undergraduate level students.
Materials and method
An online questionnaire-based survey instrument for data collection was developed using the Qualtrix software package during April/May 2020. A total of 27 questions were asked relating to students’ physical and online classroom experience, home environment, online delivery, programme of study, university location and demographics (see appendix 1). For improved validity and reliability seven response categoriesFootnote4 were provided for Likert style questions (Preston & Colman, Citation2000). The estimated completion time for the questionnaire was five minutes; the questionnaire was pilot tested in May 2020 by three students with no major issues arising.
The questionnaire was distributed by email in late May and early June 2020, shortly after the end of Spring semester. A convenience sample was taken: the survey was distributed to students on several master level programmes that the corresponding author had taught during the semester as well as, through colleagues, to students on undergraduate programmes in the same university, and, through colleagues in several international academic networks, to students in other universities in Europe, USA and Asia. A short covering letter provided the purpose of the survey, pointed out that responses were anonymous and provided contact details of the investigator. No follow-up emails were sent. As no personal details were required,Footnote5 questions were not of a contentious nature, recipients were all third level students, recipients were under no compulsion to respond, little respondent time was taken up and no incentives were provided, from an ethical point of view the survey was of low risk.
A total of 354 responses were received from students, of which 80% were fully completed. Of the respondents, 81% were undergraduate students and 19% were at master level; 59% were female, and 41% were male; 80% were in the business field; 80% were related to a university in Ireland; and 84% were aged under 25. A cumulative total of 80.5% of students reported that 50% or more of their classes were held online (N = 294). Of the class meetings held online, 81.5% of students reported that 50% or more of their classes were transmitted live (N = 292). Of the class meetings held online, 31.7% of students reported that 50% or more of their classes were pre-recorded (N = 293).
The data set was analysed using the open-source JamoviFootnote6 statistical software package. As the Likert style data collected was ordinal in nature (Jamieson, Citation2004), non-parametric testingFootnote7 was undertaken.
Results
A Wilcoxon paired samples test showed that students more strongly agreed that they were satisfied with their learning experience in the physical classroom (mean = 2.12, median = 2) than they were with their learning experience in the online classroom (mean = 3.83, median = 3) and this difference was significant (W = 3070, p < .001), supporting hypothesis H1a.
Students were satisfied with rapport with the lecturer, engagement with class materials and education technology in both the physical and online classrooms (see ). Students were satisfied with interaction with fellow students in the physical classroom (mean = 1.96, median = 2) but not in the online classroom (mean = 4.57, median = 5). Wilcoxon paired samples tests showed that: students more strongly agreed that they were satisfied with rapport with the lecturer in the physical classroom than in the online classroom (W = 2276, p < .001) supporting hypothesis H1b; students more strongly agreed that they were satisfied with interaction with fellow students in the physical classroom than in the online classroom (W = 1191, p < .001) supporting hypothesis H1c; students more strongly agreed that they were satisfied with their engagement with class materials in the physical classroom than in the online classroom (W = 3117, p < .001) supporting hypothesis H1d; students more strongly agreed that they were satisfied with educational technology in the physical classroom than in the online classroom (W = 2909, p < .001) supporting hypothesis H1e.
Table 1. Descriptive statistics.
Internet connectivity, furniture, ambience and physical environment were satisfactory for learning (mean = 3.05, 3.00, 3.69 and 2.73 respectively; median = 2, 2, 3 and 2 respectively) and Wilcoxon one-sample testsFootnote8 showed these results to be significant (p < .001, p <.001, p = 0.006, p < .001 respectively). Correlation analysis, using Spearman’s rho, of home environment variables showed that all four variables correlated with each other (all significant at p < .001) (see ). Satisfaction with home internet connection, furniture, ambience and physical environment were found to correlate with student experience of the online classroom (all significant at p < .001) supporting hypotheses H2a, b, c and d. The four home environment variables also correlated with student overall learning experience (all significant at p < .001). None of the four home environment variables correlated significantly with student learning experience in the physical classroom.
Table 2. Correlation Matrix: Learning experience and the home situation.
A Mann-Whitney independent samples test showed that master level students more strongly agreed that they were satisfied with internet connectivity (mean = 2.37, median = 2) than were undergraduate studentsFootnote9 (mean = 3.23, median = 3) and this difference was significant (U = 4485, p < .001), supporting hypothesis H3a (see ). A Mann-Whitney independent samples test showed that master level students more strongly agreed that they were satisfied with ambience (mean = 3.02, median = 3) than were undergraduate students (mean = 3.87, median = 3) and this difference was significant (U = 4788, p = .003), supporting hypothesis H3c. Satisfaction with furniture and physical environment was not found to be significantly different for master and undergraduate level students, providing no support for hypotheses H3b and H3d.
Table 3. Descriptive statistics by course level.
Table 4. Independent samples – masters and bachelors.
A Mann-Whitney independent samples test showed that master level students more strongly agreed that they were satisfied with their learning experience in the online classroom (mean = 3.26, median = 3) than were undergraduate students (mean = 3.96, median = 4) and this difference was significant (U = 4785, p = .003), supporting hypothesis H3e. Masters students were not found to be more satisfied with their overall learning experience nor with their physical classroom learning experience than were undergraduate students.
A Mann-Whitney independent samples test showed that master level students more strongly agreed that they were satisfied with interaction with fellow students in the online classroom (mean = 3.93, median = 4) than were undergraduate students (mean = 4.80, median = 5) and this difference was significant (U = 4517, p < .001). Master level students were not found to be more satisfied with interaction with fellow students in the physical classroom than were undergraduate students. Master level students were not found to be more satisfied with rapport with lecturers, engagement with class materials or with education technology in either the physical or online classrooms than were undergraduate students.
Mann-Whitney independent samples tests did not show a significant difference across gender for student learning experience overall, in the physical classroom or in the online classroom. Kruskal-Wallis tests did not show a significant difference across undergraduate year of study or field of study for student learning experience overall, in the physical classroom or in the online classroom. Kruskal-Wallis tests did not show a significant difference across age-group for student learning experience overall or in the online classroom; the test did show a significant difference across age-group for the student learning experience in the physical classroom (p = .014).
Regression analysis was carried out to examine students’ satisfaction with their learning experience in the online classroom in more detail. Three new independent variables were created by averaging the variables for the physical classroom,Footnote10 the online classroomFootnote11 and the home environmentFootnote12 (Cronbach Alphas =.736, .821, .841 respectivelyFootnote13). The dependent variable was ‘Online classroom learning experience’ (item Q4). Regression analysis results, given in , show that the learning experience in the online classroom was significantly influenced by the home environment (introduced in Model 3: ΔR2=.1539, F1, 279 = 52.97, p < .001) and by the online classroom (introduced in Model 4: ΔR2=.4472, F1, 278 = 334.60, p < .001; R2=.6322); it was not significantly influenced by the physical classroom (introduced in Model 2) or by age-group or gender (introduced in Model 1). The estimate for the online classroom predictor is .9589, much larger that the estimate for the home situation predictor at .0499.
Table 5. Linear regression (Dependent variable Q4: Online classroom learning experience).
Discussion
The pandemic provided a unique opportunity to compare the experience of the same students taking the same courses given by the same lecturers in two different learning environments, conditions akin to a controlled experiment. The results indicate that the hard work of individual academics during the pandemic, and the technical support provided by universities, largely paid off in that students were satisfied with their overall learning experience during the semester. However, students were less satisfied with their learning experience in the online classroom than they were with their learning experience in the physical classroom.
The results show that the change in the environment from physical to online classroom led to reduced student satisfaction with educational interactivity and with their learning experience, providing support for the importance of the environment for learning as set out in Bandura’s social cognitive theory. Students were more satisfied with all four modes of educational interaction (i.e. with lecturer, fellow students, course materials and technology) in the physical classroom than in the online classroom. The results suggest a strong relationship between educational interactions and the student learning experience, supporting interactivity theory. It is notable that students were not satisfied with learner-learner interaction in the online classroom, and this was especially so at undergraduate level. Given that social engagement and interactivity is important for student satisfaction, persistence and retention (Croxton, Citation2014), universities may need to introduce mechanisms to specifically increase learner-learner interaction in the online classroom. Careful design and creative thinking may be needed to achieve this, particularly for large undergraduate classes.
Students were more satisfied with their physical classroom learning experience than with their overall experience, suggesting that the online classroom portion of the semester had a strong influence on their overall satisfaction. This fits with the work of authors who suggest that student learning in the online classroom is comparable to learning in the traditional classroom (Arbaugh et al., Citation2009; Chen & Jones, Citation2007; Lyke & Frank, Citation2012) but also that students perform better in traditional than in online classrooms (Emerson & MacKay, Citation2011). Universities may need to pay especial attention to online elements of courses as these may strongly influence overall student satisfaction.
The learning experience in the online classroom was influenced both by students’ home environment and by educational interactions; however, interactions – with lecturers, fellow students, class materials and education technology – were more influential than the home environment. This is in line with Castle and McGuire (Citation2010) who suggest that course content and instructor skill are more important than delivery modality. These findings suggest that universities, when developing remote learning offerings, should draw strongly on their traditional core competency of delivering education.
Students’ home environments were satisfactory for learning, but only marginally so, and students were least satisfied with ambience in the home i.e. noise, disturbance and distraction. Strong correlations were found between the individual home environment items – internet connectivity, furniture, ambience, physical environment – and satisfaction with online learning. A correlation was found between the home environment and the online classroom, whereas no correlation was found between the home environment and the physical classroom. Universities may need to take students’ home situation more into account, especially background noise and disturbance, when designing and delivering remote courses, especially at undergraduate level. This may not be easy to do as students home situations are unknown to the university and outside its control. Considerations include availability of private study space, availability of technology such as laptop or large-size screen, internet speed and access via routers, mobile broadband or hotspots, adequacy of furniture for study, number of other people working, studying or living in the home, level of ambient noise within and outside the home. Certain actions may be considered by universities: for example, careful use of synchronous and asynchronous course elements may allow students learn at times when there is less disturbance in the home; provision of noise-cancelling headphones may reduce the impact of ambient noise.
Emergency remote education has illustrated the importance of a supportive pedagogy centred around care, affection and empathy rather than a pedagogy of teaching the curriculum (Bozkurt & Sharma, Citation2020; Goedegebuure & Meek, Citation2021; Yates et al., Citation2020). The importance of rapport with the lecturer and interaction with fellow students, found in this study, endorses such a supportive pedagogy. University workload models may need to consider additional time needed by lecturers to build rapport with remote students.
Master level students were more satisfied with their learning experience in the online classroom, and more satisfied with internet connectivity and ambience in their home environments, than were undergraduates. Master level students also were more satisfied with interaction with fellow students in the online classroom than were undergraduates. Master level students may therefore provide a better target than undergraduates for the increased use of the remote classroom.
A limitation of the study is that the forced move to remote online education during the pandemic is not necessarily representative of online learning in general as planning and design tasks were largely absent. Also, students were likely experiencing anxiety due to the pandemic itself and its associated restrictions and this may need to be considered if generalising from the results of this research. Future research could explore in more detail why masters and undergraduate students show different levels of satisfaction with online learning.
Disclosure statement
No potential conflict of interest was reported by the author.
Additional information
Notes on contributors
Malcolm Brady
Malcolm Brady is Associate Professor at the Business School in Dublin City University. He teaches business strategy, business processes and business decision making. His research is in game theory, strategic interaction and business models. He has also made several contributions to the education literature.
Notes
1. This refers to furniture for study i.e. desk, chair, shelf-space.
2. This refers to the social ambience within the home i.e. noise, distraction, disturbance.
3. This refers to the physical environment within the home i.e. light, ventilation, heating.
4. Responses were on a scale of 1 to 7, where 1 was strongly agree, 4 was neutral, and 7 was strongly disagree.
5. Other than two demographic items, gender and age range, that were used for control purposes.
6. Available at www.jamovi.org..
7. Following reviewer suggestion. See Carifio and Perla (2008), Harpe (2015), Norman (2010), Pell (2005) and de Winter and Dodou (2010) regarding use of parametric and non-parametric tools with Likert style data.
8. The test was against the neutral value of 4.
9. Note that the words undergraduate and bachelor are used interchangeably in this paper. For clarity reasons the word bachelor was used in question Q29 whereas the word undergraduate is preferred for the narrative.
10. Rapport with lecturer, interaction with fellow students, engagement with class materials, education technology (all with respect to the physical classroom).
11. Rapport with lecturer, interaction with fellow students, engagement with class materials, education technology (all with respect to the online classroom).
12. Internet connectivity, furniture, ambience and physical environment.
13. Confirmatory factor analysis showed an acceptable fit for these three factors (CFI=.948, TLI=.933 and RMSEA=.0629 and a 90% CI between .0482 and .0778; CFI and TLI are both greater than .9 and RMSEA is less than .08 as recommended by Navarro and Foxcroft (2019, p. 443). Post hoc modification increased these values to CFI=.974, TLI=.964 and RMSEA=.0461 indicating a good fit. The online classroom and home environment factors were correlated (SE = .5781); the physical classroom factor did not correlate with either the online classroom or the home environment. I thank the reviewer for suggesting factor analysis.
References
- Anderson, T. (2003). Getting the mix right again: An updated and theoretical rationale for interaction. The International Review of Research in Open and Distance Learning, 4(2), 1–14. https://doi.org/10.19173/irrodl.v4i2.149
- Arbaugh, J. B., Godfrey, M. R., Johnson, M., Leisen Pollack, B., Niendorf, B., & Wresch, W. (2009). Research in online and blended learning in the business disciplines: Key findings and possible future directions. The Internet and Higher Education, 12(2), 71–87. https://doi.org/10.1016/j.iheduc.2009.06.006
- Bandura, A. (2001). Social cognitive theory of mass communication. Media Psychology, 3(3), 265–299. https://doi.org/10.1207/S1532785XMEP0303_03
- Bandura, A. (2006). Towards a psychology of human agency. Perspectives on Psychological Science, 1(2), 164–180. https://doi.org/10.1111/j.1745-6916.2006.00011.x
- Bozkurt, A., & Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), i–vi.
- Carifio, J., & Perla, R. (2008). Resolving the 50-year debate around using and misusing Likert scales. Medical Education, 42(12), 1150–1152. https://doi.org/10.1111/j.1365-2923.2008.03172.x
- Castle, S. R., & McGuire, C. J. (2010). An analysis of student self-assessment of online, blended, and face-to-face learning environments: Implications for sustainable education delivery. International Education Studies, 3(3), 36–40. https://doi.org/10.5539/ies.v3n3p36
- Chen, C. C., & Jones, K. T. (2007). Blended learning vs. traditional classroom settings: Assessing effectiveness and student perceptions in an MBA accounting course. Journal of Educators Online, 4(1), 1–15. https://doi.org/10.9743/JEO.2007.1.3
- Coates, H., Xie, Z., & Xi, H. (2021). Engaging transformed fundamentals to design global hybrid higher education. Studies in Higher Education, 46(1), 166–176. https://doi.org/10.1080/03075079.2020.1859683
- Croxton, R. A. (2014). The role of interactivity in student satisfaction and persistence in online learning. Journal of Online Learning and Teaching, 10(2), 314–324.
- Danesh, A., Bailey, A., & Whisenand, T. (2015). Technology and instructor-interface interaction in distance education. International Journal of Business and Social Science, 6(2), 39–47.
- de Winter, J. F. C., & Dodou, D. (2010). Five-Point Likert Items: T test versus Mann-Whitney-Wilcoxon. Practical Assessment, Research, and Evaluation, 15. Article 11. https://doi.org/10.7275/bj1p-ts64
- Elshami, W., Taha, M., Abuzaid, M., Saravanan, C., Al Kawas, S., & Abdalla, M. (2020). Satisfaction with online learning in the new normal: Perspective of students and faculty at medical and health sciences colleges. Medical Education Online, 26(1). https://doi.org/10.1080/10872981.2021.1920090
- Emerson, L., & MacKay, B. (2011). A comparison between paper-based and online learning in higher education. British Journal of Educational Technology, 42(5), 727–735. https://doi.org/10.1111/j.1467-8535.2010.01081.x
- Ertl, H., & Wright, S. (2008). Reviewing the literature on the student learning experience in higher education. London Review of Education, 6(3), 195–210. https://doi.org/10.1080/14748460802489348
- Goedegebuure, L., & Meek, L. (2021). Crisis – What crisis? Studies in Higher Education, 46(1), 1–4. https://doi.org/10.1080/03075079.2020.1859680
- Harpe, S. (2015). How to analyze Likert and other rating scale data. Currents in Pharmacy Teaching & Learning, 7(6), 836–850. https://doi.org/10.1016/j.cptl.2015.08.001
- Jamieson, S. (2004). Likert scales: How to (ab) use them. Medical Education, 38(12), 1217–1218. https://doi.org/10.1111/j.1365-2929.2004.02012.x
- Kuo, Y., & Belland, B. R. (2016). An exploratory study of adult learners’ perceptions of online learning: Minority students in continuing education. Educational Technology Research and Development, 64(4), 661–680. https://doi.org/10.1007/s11423-016-9442-9
- Lyke, J., & Frank, M. (2012). Comparison of student learning outcomes in online and traditional classroom environments in a psychology course. Journal of Instructional Psychology, 39(3/4), 245–250.
- Moja, T. (2021). National and institutional responses – Reimagined operations – Pandemic disruptions and academic continuity for a global university. Studies in Higher Education, 46(1), 19–29. https://doi.org/10.1080/03075079.2020.1859688
- Moore, M. (1989). Three types of interaction. The American Journal of Distance Education, 3(2), 1–6. https://doi.org/10.1080/08923648909526659
- Navarro, D., & Foxcroft, D. (2019). Learning statistics with jamovi: a tutorial for psychology students and other beginners (Version 0.70). https://doi.org/10.24384/hgc3-7p15
- Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education, 15(5), 625–632. https://doi.org/10.1007/s10459-010-9222-y
- Pell, G. (2005). Use and misuse of Likert scales. Medical Education, 39(9), 970. https://doi.org/10.1111/j.1365-2929.2005.02237.x
- Preston, C., & Colman, A. (2000). Optimal number of response categories in rating scales: Reliability, validity, discriminating power, and respondent preferences. Acta Psychologica, 104(1), 1–15. https://doi.org/10.1016/S0001-6918(99)00050-5
- Wang, Z., Chen, L., & Anderson, T. (2014). A framework for interaction and cognitive engagement in connectivist learning contexts. International Review of Research in Open and Distributed Learning, 15(2), 121–141. https://doi.org/10.19173/irrodl.v15i2.1709
- Yates, A., Starkey, L., Egerton, B., & Flueggen, F. (2020). High school students’ experience of online learning during Covid-19: The influence of technology and pedagogy. Technology, Pedagogy and Education, 30(1), 59–73. https://doi.org/10.1080/1475939X.2020.1854337