Enhancing Human–Computer Interaction in Online Education: A Machine Learning Approach to Predicting Student Emotion and Satisfaction

Figures & data

Table 1. Related background studies for students’ satisfaction toward online learning.

Reference	Study	Model	Limitations
Masa’deh et al. (2023)	Students’ satisfaction with online learning	Structure equation modeling	Lack of generalizability due to sample from one Middle Eastern country.
Rezapour and Elmshaeuser (2022)	Impact of online learning	ML	The study focused mainly on students’ mental health.
Tran and Nguyen (2021)	Factors that affect students’ satisfaction with online learning	Structure equation modeling	The study focused on small sample size and did not identify each predictor’s contribution.
Raj and Renumol Renumol (2022)	Prediction of student engagement with online learning	ML	The study focused mainly on scores and interaction data only.
Alenezi (2022)	To analyze the social factors that influence students’ satisfaction during online learning	Structure equation modeling	The study has a small sample size and focused only on social factors.
Abdelkader et al. (2022)	Examining student satisfaction levels during Covid-19 for online learning	ML	The study employed only KNN and SVM. Also, the sample data was based on higher education.
Zapata-Cuervo et al. (2021)	Examining student’s psychological perception in online learning	Structure equation modeling	The study focused only on college students; no technical factors were analyzed with online learning.
Ho et al. (2021)	Examining students’ satisfaction during online learning	ML	Lack of generalizability due to sample from a single institution.
Lu et al. (2020)	Analyzing the factors affecting online learning acceptance	ML	There is a lack of generalizability, and ML models use default parameters, and no feature selection methods are used.

Figure 1. Methodology of the study.

Figure 2. The constructs used in this study. (a) Interaction, (b) experience, (c) health, (d) usefulness, (e) technical, (f) impact, and (g) teaching style.

Table 2. Tuning parameters for each model in satisfaction classification.

Model	Tuning parameters	Range
DT	Max-depth	2 to 22
ET	N-estimator	10–400
	Max-features	Auto, sqrt, log2
RF	N-estimator	10–550
	Max-features	Auto, sqrt, log2
LR	C	1e–04 to 1e + 02
	tol	1e–01 to 1e–06
SVM-RBF	C	2^–1 to 2^5
	Gamma	2^–2 to 2^–7
KNN	N-neighbors	3–20
	p	1–4
GNB	smoothing	1e–01 to 1e–09
ANN	Hidden layers	1–5
	Learning rate	1e–01 to 1e–06

Figure 3. Training performance of ML models based on satisfaction classification. The error bars show the standard deviation. Training is performed by a fivefold cross-validation method.

Table 3. Classification results of satisfaction prediction with feature selection with 30% unseen data.

Classifiers	Accuracy	Precision	Recall	F1-score	Friedman test rank (p value = 0.0002)
SVM	0.80	0.79	0.80	0.79	3
DT	0.80	0.79	0.80	0.79	3
KNN	0.80	0.79	0.80	0.79	3
ET	0.78	0.77	0.78	0.77	5.5
RF	0.78	0.77	0.78	0.77	5.5
LR	0.72	0.66	0.72	0.68	7.75
GNB	0.72	0.68	0.72	0.70	7.25
ANN	0.81	0.80	0.81	0.80	1

Table 4. Tuning parameters for each model in emotion classification.

Model	Tuning parameters	Range
MLkNN	N-neighbors	3–20
	smoothing	0.1–1.0
BRkNN	N-neighbors	3–20
BR	estimator-alpha	0.1–1.0
CC	estimator-alpha	0.1–1.0

Figure 4. Training performance of ML models based on emotion classification. The error bars show the standard deviation. Training is performed by a fivefold cross-validation method.

Table 5. Classification results of emotion prediction with selected features with 30% unseen data.

Classifiers	Accuracy	Hamming loss	F 1-score	Friedman’s test rank (p value = 0.9196)
MLkNN	0.615	0.215	0.541	2.6
BR	0.591	0.227	0.538	5
BRkNN	0.608	0.217	0.464	4.6
CC	0.588	0.229	0.536	6
DT	0.621	0.210	0.516	3
KNN	0.607	0.217	0.439	5.6
RF	0.653	0.194	0.549	1

Table 6. Comparison of the proposed study with related literature studies.

Reference	Outcome	Features	ML model	Result
Rezapour and Elmshaeuser (2022)	5-class	20	RF	72.58% accuracy
	3-class			83.87% accuracy
Ho et al. (2021)	7-class	16	Elastic net	65.2% explained variance
Lu et al. (2020)	3-class	16	RF	81.52% accuracy
Altrabsheh et al. (2015)	2-class emotion	–	CNB	0.47 F1-score
Proposed study	5-class	7	ANN	81% accuracy
	2-class multi-label	36	RF	0.55 F1-score

Figure 5. Feature contribution of the satisfaction prediction model.

Figure 6. Feature contribution of the emotion prediction model. (a) Bored and (b) mindfulness.

Table A1. The constructs and related questions of the study.

Constructs	Questions
Interaction	Int1: I interact with other students during class hours
	Int2: I interact with my teachers frequently during online learning
	Int3: There is no method to interact with the teachers outside the class
	Int4: The teacher’s interaction with students is good
	Int5: There is no proper parent–teacher interaction
	Int6: I have parental support at home for my studies
	Int7: My school/college provides me with the support needed during online learning
Perceived usefulness	Use1: The existing functionalities of the online education platform matched my learning needs throughout the COVID-19 pandemic
	Use2: Online learning is more attractive than traditional classroom learning
	Use3: I will still choose to use online learning if it is optional in future curriculum
Experience	Exp1: I get disconnected during live broadcasting, and this affects my comprehension
	Exp2: Sometimes, I cannot find a quiet place at home. So, I face a lack of concentration
	Exp3: I always face difficulty focusing on the computer all the time
	Exp4: I felt more isolated as a student
	Exp5: Heavy workload/homework compared to classroom learning
	Exp6: I do have disturbances from other students during online class hours
	Exp7: I get the chance to ask questions and discuss topics
	Exp8: I can achieve my targeted GPA during the online learning
	Exp9: Online learning is more challenging to adapt to than face-to-face learning
Health/social concern	Health1: My stress/anxiety level increased
	Health2: I have become lazier than before
	Health3: I often have health problems like headaches, eye strain, back pain
	Health4: I feel shy or nervous while attending online classes with the camera on
Technical problems	Tech1: Lack of gadgets for online learning in my home
	Tech2: I could not access all the materials of my course due to slow internet
	Tech3: No support is provided if any technical error occurs
	Tech4: Lack of knowledge of the use of online learning tools
Teaching style	Teach1: The course teaching method helped me understand the instructional content
	Teach2: I enjoyed the course teaching style
	Teach3: The teaching method was flexible and simple
	Teach4: The teaching method helped me improve my educational level
	Teach5: The teaching method encouraged me to study more
	Teach6: The teaching style met my expectations
	Teach7: The teaching did not include enough opportunities for discussion
	Teach8: The teaching method increased my desire to learn the subject
	Teach9: The teaching method used by the teacher-led to me achieving the course objectives well
	Teach10: The activities and examples were not sufficient to fully explain the educational content of the course
	Teach11: Overall, the teaching style was excellent
Impact of online learning	Impact1: I will not be able to return to school next term
	Impact2: My grades will be negatively affected
	Impact3: I am concerned about the time of completing or graduating the study
Concern-covid	How concerned are you about getting the coronavirus?
Satisfaction (target)	I am totally satisfied with the online learning method
Emotion (target)	The emotion you felt the most with the learning during the COVID19 pandemic. (bored, mindfulness)

Table A2. Internal reliability of the constructs.

Constructs	Cronbach’s alpha	Items
Interaction	0.723	7
Usefulness	0.924	3
Health	0.841	4
Experience	0.806	9
Technical	0.883	4
Teaching Style	0.908	11
Impact	0.774	3

Table A3. The selected features p value during the BFS method for satisfaction model.

Feature	Correlation coef.	SD error	p Value	97% confidence interval
				Lower bound	Upper bound
Tech4	−1.176	0.172	0.000	−1.512	−0.839
Teach11	−0.841	0.161	0.000	−1.157	−0.525
Impact1	1.234	0.205	0.000	0.832	1.636
Impact3	2.731	0.346	0.000	2.053	3.409
Use1	−0.406	0.166	0.015	−0.732	−0.080
Use2	−0.841	0.161	0.000	−1.157	−0.525
Use3	1.234	0.205	0.000	0.832	1.636

Table A4. The selected features p value during the BFS method for emotion model.

Feature	Correlation coef.	SD error	p Value	97% confidence interval
				Lower bound	Upper bound
Vaccinated	−0.313	0.072	0.000	−0.454	−0.172
Concern-Covid	−1.002	0.067	0.000	−1.134	−0.870
Catch-Covid	0.175	0.055	0.001	0.067	0.282
Country	−0.973	0.123	0.000	−1.213	−0.732
Gender	−0.154	0.050	0.002	−0.252	−0.055
Grade	0.968	0.250	0.000	0.478	1.458
Institution type	−0.432	0.121	0.000	−0.669	−0.196
Behavior-change	0.268	0.051	0.000	0.168	0.367
Device used	−0.499	0.097	0.000	−0.690	−0.309
OL before	−0.216	0.055	0.000	−0.324	−0.108
DL method	0.223	0.098	0.022	0.032	0.415
Computer-exp	0.489	0.078	0.000	0.335	0.643
E-library	0.152	0.067	0.024	0.020	0.284
Int1	−0.281	0.095	0.003	−0.467	−0.095
Int2	−0.272	0.104	0.009	−0.475	−0.068
Int3	−0.191	0.074	0.010	−0.335	−0.046
Int7	0.214	0.104	0.039	0.010	0.418
Exp3	0.280	0.106	0.008	0.072	0.488
Exp4	0.441	0.099	0.000	0.248	0.634
Exp5	0.194	0.084	0.021	0.030	0.359
Exp6	−0.541	0.087	0.000	−0.711	−0.371
Exp9	−0.299	0.113	0.008	−0.520	−0.078
Health1	−0.363	0.125	0.004	−0.608	−0.117
Health2	−0.501	0.126	0.000	−0.748	−0.253
Health3	0.372	0.095	0.000	0.186	0.558
Tech1	−0.250	0.099	0.011	−0.443	−0.057
Tech3	0.405	0.091	0.000	0.226	0.584
Tech4	−0.414	0.109	0.000	−0.627	−0.201
Teach2	−0.502	0.172	0.003	−0.838	−0.165
Teach4	0.419	0.151	0.006	0.122	0.716
Teach5	−0.373	0.141	0.008	−0.649	−0.097
Teach7	−0.203	0.084	0.016	−0.368	−0.038
Teach8	0.520	0.172	0.003	0.182	0.857
Teach11	−0.526	0.134	0.000	−0.788	−0.264
Impact2	−0.316	0.084	0.000	−0.480	−0.151
Use2	−0.296	0.079	0.000	−0.451	−0.141

Figure B1. The number of devices used by students for online learning.

Table B1. Demographic and other details of students.

Factor	Category	Total responses (N)	N (%)
Gender	Male	4248	46.54
	Female	4880	53.46
Grade	Graduate	601	6.58
	Undergraduate	8318	91.13
	High school	143	1.57
	Secondary school	47	0.51
	Primary school	19	0.21
Country	Kingdom of Saudi Arabia	5848	64.07
	Kuwait	2715	29.74
	Kingdom of Bahrain	547	5.99
	Sultanate of Oman	7	0.08
	United Arab Emirates	10	0.11
	Qatar	1	0.01
Institution	Public	8570	93.89
	Private	558	6.11
Distance learning method (DL method)	SOL	6024	65.99
	AOL	731	8.01
	BOL	2373	26.00
Computer Experience	Beginner	943	10.30
	Intermediate	5339	58.50
	Expert	2846	31.20
Previous Online learning experience (OL before)	No	6201	67.93
	Yes	2927	32.07
Catch corona once (catch-covid)	No	6668	73.05
	Yes	2460	26.95
Vaccinated against coronavirus	No	1797	19.69
	Yes	7331	80.31

Masa’deh, R., Almajali, D., Alrowwad, A., Alkhawaldeh, R., Khwaldeh, S., & Obeidat, B. (2023). Evaluation of factors affecting university students’ satisfaction with e-learning systems used during Covid-19 crisis: A field study in Jordanian higher education institutions. International Journal of Data and Network Science, 7(1), 199–214. https://doi.org/10.5267/j.ijdns.2022.11.003

 Google Scholar

Rezapour, M., & Elmshaeuser, S. K. (2022). Artificial intelligence-based analytics for impacts of COVID-19 and online learning on college students’ mental health. arXiv preprint arXiv:2202.07441.

 Google Scholar

Tran, Q. H., & Nguyen, T. M. (2021). Determinants in student satisfaction with online learning: A survey study of second-year students at private universities in HCMC. International Journal of TESOL & Education, 2(1), 63–80. https://doi.org/10.54855/ijte22215

 Google Scholar

Raj, N. S., & Renumol Renumol, V. G. (2022). Early prediction of student engagement in virtual learning environments using machine learning techniques. E-Learning and Digital Media, 19(6), 537–554. https://doi.org/10.1177/20427530221108027

 Google Scholar

Alenezi, A. R. (2022). Modeling the social factors affecting students’ satisfaction with online learning: A structural equation modeling approach. Education Research International, 2022, 1–13. https://doi.org/10.1155/2022/2594221

 Web of Science ®Google Scholar

Abdelkader, H. E., Gad, A. G., Abohany, A. A., & Sorour, S. E. (2022). An efficient data mining technique for assessing satisfaction level with online learning for higher education students during the COVID-19. IEEE Access, 10, 6286–6303. https://doi.org/10.1109/ACCESS.2022.3143035

 Web of Science ®Google Scholar

Zapata-Cuervo, N., Montes-Guerra, M. I., Shin, H. H., Jeong, M., & Cho, M.-H. (2021). Students’ psychological perceptions toward online learning engagement and outcomes during the COVID-19 pandemic: A comparative analysis of students in three different countries. Journal of Hospitality & Tourism Education, 35(2), 108–122. https://doi.org/10.1080/10963758.2021.1907195

 Web of Science ®Google Scholar

Ho, I. M. K., Cheong, K. Y., & Weldon, A. (2021). Predicting student satisfaction of emergency remote learning in higher education during COVID-19 using machine learning techniques. PLOS One, 16(4), e0249423. https://doi.org/10.1371/journal.pone.0249423

 PubMed Web of Science ®Google Scholar

Lu, D.-N., Le, H.-Q., & Vu, T.-H. (2020). The factors affecting acceptance of e-learning: A machine learning algorithm approach. Education Sciences, 10(10), 270. https://doi.org/10.3390/educsci10100270

 Web of Science ®Google Scholar

Altrabsheh, N., Cocea, M., & Fallahkhair, S. (2015). Predicting students’ emotions using machine learning techniques. In C. Conati, N. Heffernan, A. Mitrovic, M. Verdejo (Eds.), Artificial Intelligence in Education. AIED 2015. Lecture Notes in Computer Science (Vol. 9112). Springer. https://doi.org/10.1007/978-3-319-19773-9_56

 Google Scholar

Data availability statement

The data are not publicly available due to privacy reasons.