Student workload assessment for online learning: An empirical analysis during Covid-19

Abstract

Covid-19 has forced most educational institutions around the world to migrate to online learning in an emergency mode to protect students from the pandemic. This sudden migration to online learning has created multi-dimensional demands on students. Therefore, student workload needs to be measured during online learning. The purpose of this study is to measure the student workload from student perception by evaluating online learning in terms of Mental demand (MD), Physical demand (PD), Temporal demand (TD), Effort (EF), Performance (PE) and Frustration (FR). This study through a cross-sectional survey analysed 223 student’s workloads on six dimensions using a NASA -TLX scale. The study finds all six components of workload significant for student assessment during online learning. Besides, the NASA-TLX scale was tested using confirmatory factor analysis for its ability to assess student workload for online learning. This is the first study to assess the student workload for online learning and hence contributes to the theory of measurement of workload assessment for online learning. The educational institutions can use this study to measure the student workload assessment for various courses offered by them using this simple tool.

Keywords:

• Covid-19
• student workload
• NASA-TLX
• online learning
• mental demand
• physical demand

PUBLIC INTEREST STATEMENT

The pandemic has forced teaching and learning in educational institution to undergo a sea change by migrating to an online platform in an abrupt manner. The sudden migration to online learning environment created a multidimensional workload on the students. Traditionally student workload was measured as resource-based measure in terms of the number of learning hours in terms of lecturers, seminars etc. But for online assessment of workload warranted challenges such as connectivity, reliable power supply, availability of digital gadgets to access the online content, independent room to attend online learning, ease of loading of the website, speed of internet access, clarity of online contents and so on. Therefore, this study measures student workload from mental, physical, temporal demand, effort, performance, and frustration using NASA-TLX scale.

1. Introduction

There are reports that students are stressed about education during the Covid −19 pandemic crises (Shobhit, 2020). Corrales et al. (2020) suggest that one of the causes of student stress is overall workload. During the Covid-19 times education is shifted online because offline learning activities are not allowed in most universities. The covid-19 pandemic initiated the digital transformation of higher education. In normal circumstances, novelties in higher education would typically take many years. However, due to the pandemic, the abrupt transformation to online learning was carried out by institutions very rapidly (Adedoyin & Soykan, 2020). Online courses have two inherent issues. The first issue is in terms of macro perspective very little is established regarding the effects and efficacy of online education (McPherson & Bacow, 2015). The second is the capacity to digitally teach may differ based on a wide range of learning goals which guides the instructional/educational goals & priorities (Adnan & Anwar, 2020; Liguori & Winkler, 2020). Another aspect is the communication through human touch between the learner and the educator is absent (Dhawan, 2020). This is important because learning can be facilitated through both verbal and nonverbal communication between the learner and the educator, based on verbal or nonverbal feedback obtained from both parties during learning (Gillies, 2004; Okon, 2011). In online learning, the tendency of understanding the feedbacks may be an issue, because of the lack of human touch dimension between the learner and educator, and thus could be a challenge (Hodges et al., 2020). The users may face many technical difficulties that hinder and slow down the teaching-learning process (Favale et al., 2020). The students vary due to their capabilities, confidence level, motivation and as such some do not feel comfortable while learning online, leading to increased frustration and confusion (Dhawan, 2020). Also, the learning process requires adequate compatibility between the design technology and component of psychology of educational resources (Dhawan, 2020). However, the abrupt transition to online learning due to Covid-19 these aspects may have been neglected by the educators. Another point to consider is customization of learning process as per student’s needs, is difficult in online learning. This can create an imbalance towards students and can obstruct the learning process (Dhawan, 2020). Therefore, students have reported high levels of physical and mental stress due to online learning (Lathabhavan & Griffiths, 2020; Sahu, 2020; Son et al., 2020). Student workload is not a one-dimensional phenomenon, rather it is made up of mental demand, physical demand, temporal demand, effort, performance, and frustration. Thus, this research is important as it unearths the student workload measurement in these dimensions. The NASA-TLX tool, which is a popular tool in industrial setting, is used for the first time to measure the workload of online learning. Hence the use of this tool must be statistically validated. Thus, this research is very significant, as it will perform a confirmatory factor analysis and construct validity (Hair et al., 1998), to measure how well the proposed model measures the construct. In addition, the performance of students in online learning needs to be investigated as a function of mental demand, physical demand, temporal demand, effort, and frustration with online learning. Similarly, frustration with online learning must also to be investigated as function of mental demand, physical demand, temporal demand, effort, and performance with online learning. Thus, this study is important as it is the first study that investigates this relationship in an online learning context. The purpose of this study is to answer the research question 1) “How to multidimensionally assess the student workload for online learning during Covid-19?” 2) “Can NASA-TLX scale be used to measure student workload for online learning” 3) “What is the relationship between student performance/frustration in online learning and other dimensions of student workload”. The paper is organized as follows; the literature review is carried out briefly in the next section. It is followed by methodology, results, discussions, and conclusions. This study thus contributes theoretically to student workload assessment and performance of the student in online learning. This study will be beneficial to educators who can practically use the tool in an online context to assess the student workload.

2. Literature review

The usage of technological devices, tools and the internet for educational usage is called online learning (Means et al., 2009). One of the fundamental debates on online learning is the lack of face to face elements among the learners and teachers, and between learners (Joshi et al., 2020). Online learning is a meticulously planned activity, however, the sudden transition to online learning due to instances such as pandemics is a response to crises. This is termed as an “emergency remote teaching”, not a well-planned exercise (Hodges et al., 2020). There are multiple challenges for students to attend online classes during an emergency migration to online learning, in both developed and developing countries. In developing countries obstacles could be poor internet connectivity, unreliable power supply, limited resources like space or furniture for study, stringent lockdown measures warranting no outdoor activities (Shobhit, 2020). Online learning cannot produce desired results in underdeveloped countries where a vast majority of students are unable to access the internet due to technical as well as monetary issues (Adnan & Anwar, 2020). In developed country online learning challenges were varied. To cite an instance many teachers were unprepared to design, deliver and assess students for online learning. Besides, many teachers did not know what the virtual classroom should look like when the online classes were first started (Hannah & Strauss, 2020). Besides, the educational materials were in hard copy and conversion to soft forms, copyright issues, video conferencing software management, online hackers hacking free video conferencing sites are a challenge for teachers (Khanna & Kareem, 2021; Muthuprasad et al., 2021). Such instances result in children spending more time online rather than offline classes. The three challenges of online learning are distance, scale and personalised teaching and learning. These challenges can be further aggravated with technical issues in the website, online virtual classrooms, downloading issues, audio and video issues, installation issues, login issues, poor internet, unreliable electricity supply (Dhawan, 2020). There have been studies that suggest that online learning has been unengaging and boring (Gillett-Swan, 2017; Lee, 2020; Pawar, 2020). In addition, the student’s ability to balance, social, work, and family lives have been a challenge in the online environment (Dhawan, 2020). Consider the following scenario Student A is from rural India where one family has a mobile phone. The range is poor and therefore, interconnectivity is dismally unsatisfactory. To access pre-recorded lectures from the website or attend online classes would be a herculean challenge as internet connectivity and poor power supply will cause intermittent disruption of learning. Under these circumstances, students must spend a large amount of time in online learning. Therefore, the traditional resource-based measure of assessment of student learning will not accurately reflect the exact student workload (Therisa & Michael, 2021). In such circumstances, there would be excessive mental workload due to the cognitive content and poor accessibility (Moustafa et al., 2017). The physical demand on the student would also be excessive as most students eyes will be digitally strained due to excessive screen time and contents (Bhattacharya et al., 2020), besides suffering discomfort due to awkward body postures (Gustafsson et al., 2017). These online learning are usually time-phased, with a set of activities to be completed by a due date thus creating a time-oriented or temporal demand on the students. Besides, the effort required to achieve the level of performance would vary from student to student because students from affluent families will have access to digital resources compared to poor families (Bowles, 2018). The outcomes of the workload could be the success of the students to accomplish online learning in these circumstances. Another outcome on student workload of online learning could lead to stress for students (Molinari et al., 2005). The insecurity levels could also be high, as a student may feel left out during the online learning as all doubts may not be clarified due to lower faculty-student interaction. Therefore, there is an urgent need to test the relationship between other student workload dimensions and student performance and student frustration. Such a study will help the educators to understand the student perceived workload and how it contributes to the performance of the student and frustration level of the student towards online learning.

Student workload was measured in previous studies as a resource-based measure in terms of the number of working hours, which could consist of attending lectures, seminars or tutorials (contact hours) plus independent and private study, preparation of projects, examinations, and so forth (Kember*, 2004; Ruiz-Gallardo et al., 2011). These measures of workload were designed for offline classes. However, accessing the workload for online classes during Covid −19 warranted challenges such as connectivity, reliable power supply, availability of digital gadgets to access the online content, independent room to attend online learning, ease of loading of the website, speed of internet access, clarity of online contents and so on (Therisa & Michael, 2021). National Aeronautics and Space Administration Task Load Index (NASA-TLX) is one of the most widely used measurement tools to assess the subjective workload of individuals. It is an important multidimensional workload assessment scale, that is widely used to assess the perceived workload (Colligan et al., 2015; Hart, 2006). It is a 6 item scale and is well used in sectors such as nuclear energy, transportation, healthcare (Tubbs-Cooley et al., 2018). The scale has six dimensions of workload. Mental demand (MD), Physical Demand (PD), Temporal Demand (TD), Effort (EF), Performance (PE) and Frustration (FR). The items are further summed into a single summed unweighted score to represent the latent construct of overall workload (OW) (Byers, 1989). The OW is supposed to measure the totality of workload experienced by the individual during a specific time, event or situation (Hart & Staveland, 1988). Therefore, in this paper, we used this approach to measure the student workload of online learning on dimensions MD, PD, TD and EF, PE, and FR.

3. Methodology

3.1. Study design

A cross-sectional quantitative study was envisaged, as the NASA-TLX scale had the dimensions which could capture student workload. In addition, performance is also measured on the NASA-TLX scale and hence cross-sectional will help to test the relationship between performance and other dimensions of student workload. The present study is a cross-sectional study among Indian students studying undergraduate courses in colleges at a prominent university, in western India. The inclusion criteria of the students in this study were 1) undergraduate students 2) attending university in the year 2019–2020 3) students undergoing a regular course in an online mode due to pandemic 4) having consented to participate. The study was approved by the institutional approval board. The country has gone under lockdown since March 2020. The universities to protect the students from COVID were conducting teaching and learning online only. Online learning consists of learning through the internet, online classes, online video lectures, online self-directed learning materials, online assignments and tutorials, online examinations etc. All the participants in this study were registered students of undergraduate course of the university studying Arts, Science, Commerce, etc.

3.2. Study population & procedure

The list of students was obtained from the university and an online web-based questionnaire was randomly sent to 700 out of 1000 students. This sampling offered an equal chance for each student to be chosen. Students were informed that there is no right or wrong answer for the study, hence were instructed to respond truthfully to each question. The data was collected between 15^th July and 10 August 2020. In this study, 223 respondents participated. The response rate was 32 %. Easterby-Smith et al. (2012) argue that a 20% survey response rate is widely considered to be sufficient for a survey.

3.3. Student workload for online learning

The NASA-TLX scale was used in this study by slight modification. The respondents were asked questions to capture the perception of workload on four dimensions MD, PD, TD and EF. The question on MD was “How mentally demanding was online learning during Covid-19?” It was graded on a bipolar scale from one (very low) to 20 (very high). The question on PD was “How physically demanding was online learning during Covid-19?” and it was graded from 1 to 20. TD was measured “How hurried or hushed was the pace of online learning during Covid-19?” and graded from 1 (very low) to 20 (very high). EF was measured by “How hard did you have to work to accomplish your level of performance for online learning during Covid-19?” It was graded on a bipolar scale from one (very low) to 20 (very high).

3.4. Student performance

Student performance in this study was conceptualised as the perceived success of the student in accomplishing various online learning tasks such as examination, assignment, case studies, research paper critique and so on. Therefore, the student performance was captured using the question “How unsuccessful were you in accomplishing what you were asked to do for the online learning during Covid-19”. It was graded on a bipolar scale of 1(very low) to 20 (very high).

3.5. Student frustration with online class

Student frustration with the online class was conceptualised as the level of insecurity, discouragement, irritation, stress, and annoyance perceived by the student for the online learning activities during Covid-19. Hence, it was captured using the question “How insecure, discouraged, irritated, stressed, and annoyed were you after online learning during Covid-19?”. It was graded on a bipolar scale from one (very low) to 20 (very high).

3.6. Demographic & confounding variables

The demographic variables collected in the study included the discipline of study, age, & sex. Students who are working part-time may not find enough time to indulge in online learning and may impact the temporal demand (Hoonakker et al., 2011). Besides, part-time students may have prior work experience and may impact learning outcomes (Bayerlein, 2020). Therefore, the respondents were asked whether they work part-time or not, to detect any perceived difference in workload among full time and part-time students. As females are more committed and persistent than males, females have higher learning outcomes than males (Richardson & Woodley, 2003). Females also have better self-regulation compared to males and hence can have positive online learning outcomes than males (Alghamdi et al., 2020). Therefore, perception of student workload may vary between males and females. Hence student workload perception was tested gender wise.

3.7. Data analysis

In this study, all the analysis was conducted in Ms Excel and SPSS. Descriptive statistics, mean and standard deviation (SD) for the variable e.g., age, and percentages for categorical variables e.g., gender. The scale reliability was calculated using Cronbach alpha. Descriptive analysis of student workload was carried out gender-wise and whether student works part-time or not using t-tests. Confirmatory factor analysis is conducted to ascertain whether the proposed student workload measurement model is valid using AMOS a popular structural equation modelling software (Arbuckle, 2011). Correlation analysis is conducted among the different dimensions of the workload model to ascertain the nature of the relationship among the workload dimensions. Regression analysis was used to test the relationships between PE and MD, PD, EF, TD; and FR and MD, PD, EF, TD using SPSS.

4. Results

4.1. Descriptive analysis

The descriptive statistics of the categorical variables age are depicted in Table 1. 41% of the students were males, the mean age of the students in this study was found to be 20.36 years.,

Table 1. Descriptive statistics of the sample

Gender	Frequency	Percent
Male	92	41%
Female	131	59%
	Mean	Stdev
Age	20.36	1.895

The Cronbach alpha was calculated for the NASA-TLX scale, and it was found to be 0.890. A value above 0.7 indicates that the scale is internally consistent (Nunnally, 1994). This suggests that the scale was reliable and closely related to a set of items are as a group.

4.2. Student workload

The responses of student workload on all six dimensions are explicated in Table 2. The scoring was high on the effort dimension and low on the frustration dimension. This suggests that online learning inherently requires a student to work hard to maintain the level of performance. This could be because the physical distance between the teacher and student is getting wider due to the online learning environment and it becomes difficult to show and explain to the teacher exactly what you are struggling with (Natalie, 2020), hence the effort required by the student in online learning is higher.

Table 2. Student workload

	N	Mean	Std. Deviation
How hard did you have to work to accomplish your level of performance in your online learning during Covid-19?	223	12.95	5.921
How mentally demanding was the online learning during Covid-19?	223	12.38	5.752
How unsuccessful were you in accomplishing what you were asked to do in your online learning during Covid-19?	223	12.30	6.073
How hurried or rushed was the pace of online learning during Covid-19?	223	11.93	5.820
How physically demanding was the online learning during Covid-19?	223	11.26	5.927
How insecure, discouraged, irritated, stressed, and annoyed were you after the online learning during Covid-19	223	11.09	6.235
Valid N (listwise)	223

A t-test was conducted and no significant difference in perception of workload between males and females was observed indicating consistent perception of workload across both genders, explicated in Table 3. The effect of gender on online learning outcomes was mixed and controversial (Yu, 2021) and hence this study also did not find any difference in the perception of workload between the genders.

Table 3. Gender wise workload scores

	Gender	N	Mean	Std. Deviation	Std. Error Mean
How mentally demanding was the online learning during Covid-19?	Male	92	12.57	5.675	.592
	Female	131	12.24	5.823	.509
How physically demanding was the online learning during Covid-19?	Male	92	11.13	5.984	.624
	Female	131	11.36	5.908	.516
How hurried or rushed was the pace of online learning during Covid-19?	Male	92	12.13	5.702	.594
	Female	131	11.79	5.919	.517
How unsuccessful were you in accomplishing what you were asked to do in your online learning during Covid-19?	Male	92	11.91	6.076	.633
	Female	131	12.58	6.079	.531
How hard did you have to work to accomplish your level of performance in your online learning during Covid-19?	Male	92	12.91	5.629	.587
	Female	131	12.98	6.139	.536
How insecure, discouraged, irritated, stressed, and annoyed were you after the online learning during Covid-19	Male	92	10.48	5.805	.605
	Female	131	11.51	6.507	.569

The workload scores were also calculated between students who work part-time and those who do not work part-time. The workload scores are given in Table 4

Table 4. Workload scores based on student

Do you work on a part time basis along with your studies?		N	Mean	Std. Deviation	Sig (2 tailed test)
How mentally demanding was the online learning during Covid-19?	Yes	59	11.59	6.111	0.22
	No	164	12.66	5.609
How physically demanding was the online learning during Covid-19?	Yes	59	10.71	5.580	0.40
	No	164	11.46	6.051
How hurried or rushed was the pace of online learning during Covid-19?	Yes	59	11.39	5.887	0.41
	No	164	12.12	5.801
How unsuccessful were you in accomplishing what you were asked to do in your online learning during Covid-19?	Yes	59	12.41	5.974	0.88
	No	164	12.27	6.126
How hard did you have to work to accomplish your level of performance in your online learning during Covid-19?	Yes	59	12.95	6.185	0.99
	No	164	12.95	5.842	0.99
How insecure, discouraged, irritated, stressed, and annoyed were you after the online learning during Covid-19	Yes	59	9.63	5.515	0.04
	No	164	11.61	6.409

A t-test was conducted between both groups, and it was found that only frustration level was lower among students who did part-time work. This could be because the part-time students by their work experience were better at handling the insecurity, discouragement, irritation and annoyance because of cross-transferability of skills (Snell et al., 2016), these students were less frustrated compared to others. Other dimensions did not factors did not significantly differ between those who worked part-time and those who did not work.

4.3. Confirmatory factor analysis

To test the conceptually grounded NASA-TLX to measure the student workload of online classes during Covid-19, confirmatory factor analysis was conducted using AMOS 18. CFA specifies how measured variables logically and systematically represent the construct involved in the theoretical model. The figure depicts the first-order confirmatory model along with the standardised factor loading. The model fit indices of the model Chi-square was value 9.410, df = 6, P = 0.152, SRMR = 0.0196, GFI = 0.986, CFI = 0.9962, RMSEA = 0.051. The model fit indices are within the acceptable limits (Hair et al., 2014; Tabachnick & Fidell, 2007), indicating the appropriateness of the specified model. The standardised loading estimate is above 0.5 and therefore it depicts construct validity (Hair et al., 1998). The Figure 1 depicts the confirmatory factor analysis.

Figure 1. Confirmatory factor analysis.

The Average variance extracted was calculated was found to be 0.5821, indicating convergence. The construct reliability was calculated using the formula suggested by (Hair et al., 1998) and it indicates convergent validity.

$CR=\frac{{\sum }_1^n{L}_i}{{\sum }_1^n{L}_i+{\sum }_1^n{e}_i}$

L_i = sum of factor loading and e_i = error variance.

It was found to be 0.7 indicating good reliability and hence convergent validity is supported and suggesting measures all consistently represent the same construct.

4.4. Correlation analysis of student workload dimensions

To understand the relationship between the six dimensions of student workload a bivariate Pearson correlation is carried and is explicated in Table 5.

Table 5. Correlation between workload

		How mentally demanding was the online learning during Covid-19?	How physically demanding was the online learning during Covid-19?	How hurried or rushed was the pace of online learning during Covid-19?	How unsuccessful were you in accomplishing what you were asked to do in your online learning during Covid-19?	How hard did you have to work to accomplish your level of performance in your online learning during Covid-19?	How insecure, discouraged, irritated, stressed, and annoyed were you after the online learning during Covid-19
How mentally demanding was the online learning during Covid-19?	Pearson Correlation	1	.744**	.716**	.589**	.698**	.426**
	Sig. (2-tailed)		.000	.000	.000	.000	.000
	N	223	223	223	223	223	223
How physically demanding was the online learning during Covid-19?	Pearson Correlation	.744**	1	.590**	.541**	.655**	.440**
	Sig. (2-tailed)	.000		.000	.000	.000	.000
	N	223	223	223	223	223	223
How hurried or rushed was the pace of online learning during Covid-19?	Pearson Correlation	.716**	.590**	1	.506**	.688**	.503**
	Sig. (2-tailed)	.000	.000		.000	.000	.000
	N	223	223	223	223	223	223
How unsuccessful were you in accomplishing what you were asked to do in your online learning during Covid-19?	Pearson Correlation	.589**	.541**	.506**	1	.661**	.327**
	Sig. (2-tailed)	.000	.000	.000		.000	.000
	N	223	223	223	223	223	223
How hard did you have to work to accomplish your level of performance in your online learning during Covid-19?	Pearson Correlation	.698**	.655**	.688**	.661**	1	.582**
	Sig. (2-tailed)	.000	.000	.000	.000		.000
	N	223	223	223	223	223	223
How insecure, discouraged, irritated, stressed, and annoyed were you after the online learning during Covid-19	Pearson Correlation	.426**	.440**	.503**	.327**	.582**	1
	Sig. (2-tailed)	.000	.000	.000	.000	.000
	N	223	223	223	223	223	223

**. Correlation is significant at the 0.01 level (2-tailed).

The unsuccess of accomplishing what one was asked to do during online learning increases when the online learning is mentally demanding, physically demanding, temporarily demanding, the effort required is high and frustration is high. In other words, the student performance in online classes decreases when the online class MD, PD, TD, EF and FR is very high. Student performance in online classes is the combination of different scores such as assignments, theory exams, practical exams, group projects, online discussion etc (H. C. Wei & Chou, 2019; H.-C. Wei & Chou, 2020). Each of these tasks requires varying degrees of MD, PD, TD, EF and FR. Higher these demands the student may not be able to cope in an online setting, because the student to teacher interaction based on feedback is minimal in an online learning environment (Elberkawi et al., 2021). The frustration level increases when online learning is mentally demanding, physically demanding, temporarily demanding, the effort required is high and success is low. In other words, the insecurity, discouragement, annoyance, stress will increase when MD, PD, TD, EF required for a task is high. Further, the frustration will increase when their teacher-student interaction and peer-to-peer interaction among the students are low (Lai et al., 2019), as in the case of online learning. MD, PD, TD and EF are highly positively correlated and significant at 5% level. This depicts that in an online learning environment equal workload weightage should be given to these dimensions. An increase in workload in one dimension will increase the impact on the other due to the interaction effect and hence educators should carefully consider these aspects while designing the course outline.

4.5. Regression analysis

To investigate the relationship between the frustration level of the student with online with MD, PD, TD and EF an ordinary least square multiple regression was employed. The regression statistics are explicated in Tables 6 and 7.

$FR=2.936+0.\hspace{0.17em}221TD+5.284\hspace{0.17em}EF$

Table 6. Regression statistics FR with MD, PD, TD and EF

Model Fit	R2 = 0.354 F = 31.412 Sig = 0.000
	Unstandardized Coefficients		Standardized Coefficients	t	Sig.
	B	Std. Error	Beta
(Constant)	2.562	.873		2.936	.004
How mentally demanding was the online learning during Covid-19?	−.138	.105	−.127	−1.322	.188
How physically demanding was the online learning during Covid-19?	.119	.089	.113	1.346	.180
How hurried or rushed was the pace of online learning during Covid-19?	.237	.089	.221	2.651	.009
How hard did you have to work to accomplish your level of performance in your online learning during Covid-19?	.468	.089	.445	5.284	.000

Dependent variable: How insecure, discouraged, irritated, stressed, and annoyed were you after the online learning during Covid-19

Table 7. Regression statistics PE with MD, PD, TD and EF

Model Fit	R2 = 0.463 F = 48.787 Sig = 0.000
	Unstandardized Coefficients		Standardized Coefficients	t	Sig.
	B	Std. Error	Beta
(Constant)	2.544	.775		3.281	.001
How mentally demanding was the online learning during Covid-19?	.217	.093	.206	2.341	.020
How physically demanding was the online learning during Covid-19?	.092	.079	.090	1.173	.242
How hurried or rushed was the pace of online learning during Covid-19?	−.019	.079	−.018	−.241	.810
How hard did you have to work to accomplish your level of performance in your online learning during Covid-19?	.483	.079	.471	6.138	.000

Dependent Variable: How unsuccessful were you in accomplishing what you were asked to do in your online learning during Covid-19?

The regression equation suggests that student frustration with online learning is thus dependent on temporal demand and efforts. If the workload on temporal demand is high students may find it difficult to complete the various tasks (Batu et al., 2018; Diamantes, 2007; Lemay & Doleck, 2020) in an online learning environment. To cite an instance if the activities are designed to be completed in a fast-paced manner without considering the temporal demand of workload, the chances of completion of online learning tasks will be low thus increasing the frustration levels. Similarly, to understand the relationship between unsuccess in accomplishing what one was asked to do during online learning with MD, PD, TD and EF an ordinary least square multiple regression was employed. The regression equation will help us to clarify the most significant contributing factor.

$PE=3.281+\hspace{0.17em}2.341MD+\mathrm{6.138.}\hspace{0.17em}EF$

The regression equation suggests that the student performance in online learning is dependent on the mental demand of the student workload and the effort required. If the tasks are mentally demanding and require high efforts, the motivation for the completion of tasks will be low (Capa et al., 2008). Online learning requires assessing video lecturers, attending online lectures, assignments, lecture notes etc. If the motivation is low students will not be motivated to use these resources and resulting in poor student performance.

The R-squared value of models is 0.354 and 0.463. The low value of R-square is quite common for cross-sectional analysis (Wooldridge, 2016). In cross-sectional studies, a value of R-squared above 0.2 is usually considered decent (Harper et al., 2005). Ashenfelter and Krueger’s report R-squared in the range of 0.2 and 0.3, with a sample size of 298 (Krueger & Ashenfelter, 1992). Levitt reports R-squared in the range of 0.06 and 0.37 with a sample size between 1,276 and 4,801 (Levitt, 1995). Therefore, this model is meant to explanatory and confirmatory rather than being predictive.

5. Discussion

This study evaluated the student workload during Covid-19 pandemic online classes. The results depict that most of the students had to work hard to accomplish the level of performance in online learning during Covid-19 and it was ranked by the students at the number one position. Online learning is a relatively new phenomenon in India (Jahangeer, 2020). The pandemic warranted an abrupt transition to online learning in an emergency manner and authorities & students were ill-prepared (Kumar, 2020). Therefore, the effort required to achieve the performance was on the higher side.

Students have ranked that online learning was mentally demanding during Covid-19 at the second position. In online learning mental activity becomes dominant and learning is aimed at meaning-making, including attending to relevant material, mentally organizing it into the distinct concept (Mayer, 2019). This happens with the lesser student to teacher interaction than offline in an Indian context, because of resource constraints such as data shortage, poor internet connectivity, irregular power supply etc (Sharma, 2020). Therefore, students have perceived online learning to be mentally very demanding during the lockdown period.

In the third position, students felt they were mostly unsuccessful in accomplishing the online learning activity during Covid-19. These students were coming from a background where a digital resource in education was a luxury rather than a necessity, as most universities were beginning to adopt online learning (Jahangeer, 2020). Completing an online learning activity requires remembering, understanding, applying, analysing, evaluating and creating. For this to happen, there must be adequate resources to both acquire, apply, and create knowledge. Pandemic has hampered the student’s needs to acquire the above, due to lockdown as resources were limited. A study suggests that some 23.8 million additional children and youth (from pre-primary to tertiary) may drop out or not have access to school next year due to the pandemic’s economic impact alone (United Nations, 2020). The design of online learning in a developing country like India must consider resource availability into consideration while designing online content so that students can be successful at it.

At the fourth position, students have perceived that online learning was hurried or rushed during Covid-19. This may be because initially the lockdown was announced when the country was unprepared (Lancet, 2020). Even educational institutes were not prepared to carry out online learning (Shobhit, 2020). So, it took time for these institutes to gear up to online learning and besides, the pressure of academic year competition might have led to the hurried pace of online learning. At fifth position was student’s perception that online learning was physically demanding during Covid-19. Most students in India access online content using handheld mobile devices (Shobhit, 2020). Online learning creates a physical strain on the eyes called digital eye strain when these small digital screens are used (Bhattacharya et al., 2020). Besides, online learning can also cause symptoms of MSD leading to physically tiredness (Amro et al., 2020). Thus, students felt online learning is physically demanding.

At the sixth position, students felt insecure, discouraged, irritated, stressed, and annoyed towards online learning during Covid-19. The epidemic has brought not only the risk of death from infection but also unbearable psychological pressure. A study in China posits that economic effects, and effects on daily life, as well as delays in academic activities, were positively associated with anxiety symptoms (Cao et al., 2020). Besides, a growing number of universities across the world have either postponed or cancelled all campus events such as workshops, conferences, sports, and other activities (Sahu, 2020). Without these activities, students are frustrated and stressed, and coupled with the stress of online eLearning has created stress and annoyance towards online learning. It was further found in this study that there was no difference in perception of workload between males and females. This depicts that both genders have uniformly evaluated the workload. The perception of workload among students who worked part-time and those who did not only differed on frustration level with online learning. Those who did not do part-time work were more frustrated than those who did part-time work. Students who do part-time work will have some advantages in terms of money, resources and besides, they will also pick up various generic cross-functional skills to carry out complex tasks (Billett & Ovens, 2007; Kember et al., 2001). Therefore, part-time students might have applied these learned skills over time in part-time work such as handling stress etc for dealing with online learning during the pandemic, which helped them to be less frustrated than others who did not work. The confirmatory factor analysis confirms the student workload measure the variables logically and systematically to represent the construct involved in the theoretical model. This suggests that the NASA-TLX model should be used to measure student workload for online classes. This will help the online course designers to consider the six dimensions for an effective online course design.

The dimensions of student workload are significantly correlated, it indicates that if one increases the other also increases. The online workload should be carefully designed by considering all dimensions. Student performance is one of the key areas in online learning. The regression analysis reveals that students will be unsuccessful in activities of online learning if the student workload is mentally demanding, and the effort required is very high. This result will help the course designers to evaluate each online activity in terms of mental demand and efforts required for competition. If careful consideration is given to both elements, the chances of success of students in online learning will be high. The frustration of students with online learning can be significantly explained by temporal demand and efforts. With the hushed-up or fast-paced online learning, the frustration level of the students will be high. Similarly, if the effort required for carrying out the activities are very high, the frustration level of students with online activity increases.

The study contributes to the theory of workload measurement of students in online learning. Further, the application of the NASA-TLX scale was used for the first time in student workload measurement for online learning. The CFA suggests that the NASA-TLX can be used as a measurement model. Educators can use this tool to measure the student workload for their courses. This tool will further help the educators to consider the multi-dimensional student workload while designing the courses. It will help the educators to calculate proper credits to these courses so that students are benefitted based on online learning.

6. Limitation and future work

The study used the NASA-TLX self-reported scale to measure the student workload. Socially desirable responding (Grimm, 2010) is one of the main issues in the self-reported questionnaire, however, in this study questionnaire was made anonymous without incorporating any personal identifying details to account for socially desirable responses. However, future studies should measure student workload using objective measurement for comparing the results using both objective and subjective data. To understand the time-oriented variation of workload a longitudinal study may be undertaken. As this study was conducted in India and the challenges were unique such as irregular power supply, poor internet facilities, digital devices shortage etc (Jahangeer, 2020) and hence perception of workload may also vary. Similar studies should be conducted in developed countries to compare student perception of workload, as some of the challenges of the developing country will be non-existent in a developed country. Future studies may also assess the workload course wise, to ascertain whether the workload varies differently among varied courses. Another area of study would be that the results obtained in this study must be corroborated with other data collection techniques such as interviews so that it strengthens the finding of this study.

7. Conclusion

This study was intended to assess the student workload during Covid-19 online learning. NASA-TLX scale was used to assess the student workload multidimensionally on MD, PD, TD, EF, PE, and FR dimensions. The confirmatory factor analysis, construct reliability, reliability analysis suggests that NASA-TLX can be statistically used to measure student workload for online learning. The multiple regressions suggest that student performance in online learning is found to be related to efforts and mental demand. Also, the frustration with online learning is related to temporal demand and efforts. Further, the student who was working part-time showed lesser frustration levels with online classes than others

Acknowledgements

We are extremely thankful to all the respondents who participated in this study despite the pandemic. We also thank the editor and four reviewers for the constructive comments, which we believe have improved the paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The authors received no direct funding for this research.

Notes on contributors

Michael Sony

Lt. (Dr) Karingada Kochu Therisa Beena is Assistant Professor in Zoology at DCT’s Dhempe College of Arts & Science, Goa India has been researching in the areas of zoology. She also researches in designing teaching materials, online learning and assessments. She is also interested in the application of ergonomics in the educational settings. Dr. Michael Sony is a Senior Lecturer in Industrial Engineering at Namibia University in Science and Technology, Windhoek, Namibia, and he researches in production and operations management, Industry 4.0, and teaching & learning during fourth industrial revolution. The study was designed by the authors as a part of their study on online learning during covid. This is an ongoing study undertaken by the authors. KKTB and MS conceived the ideas; KKTB collected the data; KKTB and MS analysed the data, and KKTB led the writing.