Enhancing Human–Computer Interaction in Online Education: A Machine Learning Approach to Predicting Student Emotion and Satisfaction

Abstract

Maintaining student engagement is pivotal in the rapidly evolving landscape of online education. The factors that affect and predict students’ satisfaction during online learning are poorly understood. Our study addresses this issue by employing machine learning (ML) models to predict student emotions and satisfaction. ML focuses on small and structured data that do not require the complexity of deep neural networks. We created an ML classification model trained with data from an online survey of seven constructs. We narrowed down the critical features from 55 to 7 and 36 for satisfaction and emotion prediction, respectively, using the backward feature selection (BFS) method. The artificial neural network (ANN) and the random forest (RF) model outperformed other classifiers, showing 81% and 65.3% accuracy for satisfaction and emotion prediction, respectively. Our findings suggest that by predicting and responding to student emotions and satisfaction, we can optimize the online learning experience, providing personalized educational trajectories that align with each student’s unique needs.

Keywords:

• Artificial intelligence
• feature selection
• machine learning
• online learning
• student satisfaction

View correction statement:

Correction

1. Introduction

A global pandemic caused by the coronavirus (COVID-19) erupted in 2019, and the World Health Organization (WHO) recommends that all educational institutions transition away from traditional classroom instruction and toward web-based e-learning or online distance learning, which helps decrease virus reproduction (Zu et al., 2020). On a large scale, online education has fast become an attractive mode of education delivery (Elfaki et al., 2019). Due to the development of technology, learning is now regarded as adaptive, and educators must be computer-advanced to support their students (Ally, 2019). Learning technology has mitigated the pandemic, but colleges’ transition to online education is difficult. Online instructional programs are poorly planned, designed, and developed. Students and faculty may lack the digital skills needed to succeed online. Educational technology researchers must create plagiarism-free assessment methods. Despite the obstacles, online learning will become increasingly hybrid in the future (Adedoyin & Soykan, 2023).

The value of the learning experience contributes to student satisfaction. Students’ satisfaction with their learning experience determines the success of online education (Azizan et al., 2022). According to Donohue and Wong, the level of satisfaction among students can influence their motivation (Donohue & Wong, 1997). Artificial intelligence (AI), including ML and deep learning (DL), will massively affect education, and many studies evaluate student performance using different ML models. Lecturers can better understand learning and teaching using AI techniques, and learning systems are more flexible and easier to design to help learners learn and gain new skills and insights (du Boulay, 2016). Based on student learning behavior, a system for predicting student course satisfaction was prepared by Tzeng et al. (2022). The studies should emphasize the importance of establishing policies and strategies to ensure that schools and universities can efficiently utilize AI technologies while adhering to a human-centered approach to online learning (Dogan et al., 2023).

One must assess the tools and learning materials used to use online learning effectively. The institution should understand how well the learner, educators, and content (LEC) requirements are met (Mutawa, 2020). The vital parameter for measuring the success of education, whether online or remote, is student satisfaction (X. Li & Asante, 2021). Identifying the factors affecting learning effectiveness helps the educator design the course content (Kauffman, 2015). A student’s emotional state also plays a vital role in measuring their education success (Altrabsheh et al., 2015). A comprehensive analysis demonstrates that transmitting content via online mediums is successful regarding learning outcomes and achievements. The research on whether or not online learning is helpful is inconclusive because it is heavily impacted by developing digital technology (Batdı et al., 2021). Wang et al. (2023) study reveals that the information, system, and service quality improved user satisfaction, communication quality, loyalty intention, and learning efficacy, while the monitoring quality alone affects communication.

By evaluating linear and non-linear correlations, ML constructs models that use categorical and numeric factors (Alsariera et al., 2022; Bhutto et al., 2020; Kaddoura et al., 2022). The common ML models are SVM, LR, DT, RF, KNN, Naive Bayes (NB), and artificial neural network (ANN) (Alnagar, 2020; Dejaeger et al., 2012; Permana et al., 2017; Skrbinjek & Dermol, 2019). Even though most studies use a classification method, clustering was also used to categorize students’ success rates using online learning (Saputra et al., 2021). An SVM classification algorithm was used to classify these clusters. Some ML models are used to predict students’ learning styles, which helps understand the concept in depth (Rasheed & Wahid, 2021).

Ho et al. (2021) implemented a model with ML regression algorithms, which shows content delivery, the effort of educators, and preference for traditional learning as significant predictors. With the help of the clustering algorithm, the student’s choice of learning method was predicted (Lu et al., 2020). Rincon-Flores et al. (2022) studied the learning profiles of individual students with learning analytics. Their findings suggest that the accuracy of the forecasts increases as more data are added to the model, and the teaching-learning process improves with reduced academic failure.

The emotion of students also affects their learning outcomes. ML model helps personalize the course content according to their emotional states (Liu & Ardakani, 2022). Different ML models, like multi-label and multi-task, are used in the study conducted by Gibson et al. (2022) based on human behaviors. To make the model more efficient, the feature selection method is essential. The feature selection algorithm aims to improve the model accuracy and reduce dimensionality (J. Li et al., 2017).

One of the most important steps in ML is called "feature selection." During this stage, a subset of pertinent features is chosen from the whole collection to feed into the model. Feature selection aims to boost the model’s performance and interpretability and to limit the risk of overfitting by picking the relevant features (Lahbib et al., 2023). The filter and wrapper are the two approaches to performing feature selection. Wrapper techniques include feature selection and classification model interaction, resulting in greater prediction accuracy values than filter approaches (Zhu & Song, 2013). Backward feature selection (BFS) is a wrapper method that starts with all features and gradually removes the least significant ones, making the final selected features easier to understand.

Table 1 shows relevant studies on the satisfaction of students during online learning. Most related studies focused on student satisfaction based on a particular course, data collection from a single institution or school, and mostly on college students. AI has changed education, social interactions, and tradition. The study by Arpaci et al. (2022) has evaluated individuals’ perspectives on the social sustainability of the Metaverse employing structure equation modeling with ANN. It lacks generalizability as it collects data from a particular people in a country. Different studies have employed a hybrid model with structure equation modeling and ANN (Akour et al., 2022; Lo et al., 2022).

Table 1. Related background studies for students’ satisfaction toward online learning.

Reference	Study	Model	Limitations
Masa’deh et al. (2023)	Students’ satisfaction with online learning	Structure equation modeling	Lack of generalizability due to sample from one Middle Eastern country.
Rezapour and Elmshaeuser (2022)	Impact of online learning	ML	The study focused mainly on students’ mental health.
Tran and Nguyen (2021)	Factors that affect students’ satisfaction with online learning	Structure equation modeling	The study focused on small sample size and did not identify each predictor’s contribution.
Raj and Renumol Renumol (2022)	Prediction of student engagement with online learning	ML	The study focused mainly on scores and interaction data only.
Alenezi (2022)	To analyze the social factors that influence students’ satisfaction during online learning	Structure equation modeling	The study has a small sample size and focused only on social factors.
Abdelkader et al. (2022)	Examining student satisfaction levels during Covid-19 for online learning	ML	The study employed only KNN and SVM. Also, the sample data was based on higher education.
Zapata-Cuervo et al. (2021)	Examining student’s psychological perception in online learning	Structure equation modeling	The study focused only on college students; no technical factors were analyzed with online learning.
Ho et al. (2021)	Examining students’ satisfaction during online learning	ML	Lack of generalizability due to sample from a single institution.
Lu et al. (2020)	Analyzing the factors affecting online learning acceptance	ML	There is a lack of generalizability, and ML models use default parameters, and no feature selection methods are used.

Numerous studies cited in the literature have faced criticism due to their restricted applicability to broader contexts. This constraint emerges due to the reliance on data and conclusions frequently obtained from a specific place, institution, or student group. Research with limited sample numbers may not possess sufficient statistical power to identify minor effects or correlations reliably. Moreover, it should be noted that the conclusions drawn from small sample sizes may not accurately reflect the characteristics and behaviors of the broader community of individuals engaged in online learning. Moreover, it is worth noting that specific ML algorithms may exhibit superior performance in different scenarios. Technical concerns, including platform usability, internet connectivity, and resource access, can strongly impact the online learning experience. However, these factors are given little attention.

The proposed study addresses this research gap by incorporating the six Gulf Cooperation countries’ primary, secondary, high school, graduate, and undergraduate students. Also, we analyzed seven factors related to online learning that affect students’ satisfaction. Another direction of the study is based on emotional prediction during learning, which will help the instructor to understand the student’s engagement in class. The proposed study employs an ANN neural network model with four hidden layers. The study aims to predict satisfaction with online learning and the student’s emotional state of mind during the learning process using ML.

Two classification model is implemented based on students’ data collected through a questionnaire. The first is a multi-class ML model for predicting student satisfaction with online learning. Furthermore, the other is for emotional state prediction during the learning process (bored and mindfulness) and is a multi-label ML model. There are only a few studies based on the multi-label classification model. This work uses the following ML models: logistic regression (LR), decision tree (DT), extra tree (ET), support vector machines (SVMs) with radial basis function (RBF) kernel, K-nearest neighbors (KNNs), Gaussian Naïve Bayes (GNB), and random forest (RF) algorithms. We employed a feedforward neural network model with few dense layers (ANN) for the satisfaction classification task. Models such as multi-label K-nearest neighbor (MLkNN), binary relevance (BR), KNN, RF, binary relevance K-nearest neighbor (BRkNN), classifier chain (CC), and DT are applied for multi-label classification (Sorower, 2010). The features are selected based on the BFS method and then compared with the complete ML models.

The following are the key contributions of the study:

• The study captures the complexity of Arab students’ emotions and levels of satisfaction with their online education experiences using multi-class and multi-label classification models. The students belong to primary, secondary, high school, graduate, and undergraduate.

• The proposed study employs different ML and a neural network model for satisfaction and emotion classification problems.

• There has been limited research on multi-label classification in online learning, a research gap. By using and comparing MLkNN, BR, BRkNN, CC, KNN, RF, DT, and ANN, the study fills in this gap and helps to improve knowledge in this area.

• The BFS technique is employed in this study to choose significant factors that contribute considerably to predicting student satisfaction and emotional states during online learning. The use of BFS in this context is a novel component of the study.

• The study’s findings apply to educational institutions and online learning platforms. Understanding the significant elements influencing student satisfaction and emotions enables data-driven decision-making to improve the online learning experience.

Accordingly, the main research questions (RQs) discussed in this work are:

RQ1: Which model best predicts students’ satisfaction and emotional states in online learning?

RQ2: What key features contribute to student satisfaction and emotion prediction in online learning?

The remainder of the article is organized as follows: the materials and methods are specified in Section 2, which describes the ML models and evaluation matrices, followed by the result and discussion in Sections 3 and 4, respectively. Finally, the significance of the work is concluded in the last section.

2. Materials and methods

The methodology of the work is described in Figure 1. Collecting the student’s data based on the questionnaire is the primary stage, followed by data preprocessing. It transforms the collected data into a standard scale suitable for the ML models to work. The data set is split into training and testing with a ratio of 70:30, where 30% is set as unseen data. The BFS method is applied to 70% of the training data to select the best-suited features for the ML models. Different multi-class and multi-label ML models are selected to predict students’ satisfaction and emotion, respectively. Each of these ML model’s parameters is chosen based on hyperparameter tuning with the help of the GridSearch method. The training data are split up into fivefold for performing fivefold cross-validation. First, the model is trained on four different folds and then evaluated on the remaining fold. This procedure is repeated five times until all folds are categorized as test folds. The model’s average performance is chosen and saved for testing the new unseen data. It ensures that the model is not overfitting the training data. The detailed information of each stage is explained in the following sub-sections.

Figure 1. Methodology of the study.

2.1. Dataset description

An online questionnaire survey is being disseminated to schools and colleges in the Gulf Cooperation Council (GCC) countries by the Secretariat of the E-learning Officials Committee at Arab Gulf Cooperation Universities and Higher Education Institutions. The survey collects demographic information such as gender, grade, nationality, institution, computer familiarity, and categorical characteristics such as learning method, vaccination details, coronavirus concerns, behavioral changes, and learning equipment. Additionally, there are multiple-choice questions and Likert scale questions in the questionnaire. The data were collected between June and December 2021 via online forums, with 18,007 impression clicks and 9194 completed surveys submitted by students from various schools, colleges, and universities throughout the GCC six countries. These countries are the Kingdom of Bahrain, the Kingdom of Saudi Arabia, the Sultanate of Oman, the United Arab Emirates, Kuwait, and Qatar. The topic of the study was likely the primary factor that had a role in the choice to include individuals from the GCC nations in the survey.

The survey included seven constructs: interaction, usefulness, technical, teaching style, online learning experience, the impact of learning, and health concepts. The survey ended with a question on satisfaction and the emotional states experienced by online learning. After cleaning the data, the total count was 9128. Table A1 describes the constructs and associated questions. The concern of students toward covid-19 (concern_covid) is also asked during the survey. All the Likert-scale questions and the concern_covid are from zero to four (strongly disagree to agree strongly). Cronbach’s alpha value is analyzed for the internal consistency between the constructs. The alpha value ranges from 0 to 1, where the closest to 1 denotes more reliability. The rule is that the reliability is acceptable if the value is greater than or equal to 0.7 (George & Mallery, 2021). All values in Table A2 are higher than 0.7, which indicates that the survey questions are reliable. The mean and standard deviation (SD) of all seven constructs are depicted in Figure 2.

Figure 2. The constructs used in this study. (a) Interaction, (b) experience, (c) health, (d) usefulness, (e) technical, (f) impact, and (g) teaching style.

2.2. Feature selection

The feature selection method, BFS, is a wrapper method (Mehmood et al., 2022). There is a possibility that ML models cannot make accurate predictions from the dataset due to the high dimensionality of features. The BFS algorithm commences by initializing a model that encompasses all available features. It then proceeds to iteratively eliminate the features deemed to have the least significance, as determined by a predefined threshold, namely a p value greater than 0.05 (Aggrawal & Pal, 2021). The consideration of the influence of each feature in the presence of others is crucial, particularly in cases when features exhibit interactions or dependencies. BFS has the potential to offer computational efficiency over forward selection by initially employing a comprehensive model and subsequently removing features. This advantage becomes especially significant when presented with an adequate number of features.

Calculating the p value for the BFS method was done using ordinal LR, and the significant value was set as 0.05 (Warner, 2008). According to the null hypothesis, the other variables selected will not affect the target variable, which is accepted when the p value is higher than 0.05. The features are eliminated when p value > 0.05. The chosen feature p value is mentioned in Table A3 (satisfaction model), and Table A4 (emotion model).

2.3. Classification models

The data were cleaned (removing any empty data, in case) and pre-processed using Python and Scikit-learn ML software (Pedregosa et al., 2011). In a ratio of 70:30, the whole dataset is divided into training and testing. Seventy percent of the data is cross-validated five times, and 30% is set for external validation based on unseen data. The multi-class models used to predict online learning satisfaction are DT, KNN, SVM, RF, ET, LR, GNB, and ANN. These classification models were chosen because they are widely accepted. The final target class is the satisfaction variable, which ranges from 0 to 4. The best parameter was estimated using the grid search method. With the best estimator, the data were trained and validated with the stratified fivefold method. The remaining 30% of the data are tested with the best parameter obtained from each model. Precision, recall, F1-score, and accuracy matrices are evaluated for multi-class models (Hossin & Sulaiman, 2015).

All the models used here are supervised classification algorithms. DT is a tree-based algorithm, including root, internal, and leaf nodes (Pallathadka et al., 2023; Skrbinjek & Dermol, 2019). The KNN algorithm performs the learning based on KNNs. The value of k is obtained by tuning the parameter (Liu & Ardakani, 2022). The RF and ET are ensemble classifiers that fit many DTs, each constructed from sub-samples (Iqbal & Varshney, 2022; Raj & Renumol Renumol, 2022). LR is a linear classification model. SVM handles high-dimensional data and finds a hyperplane that fits the data into distinct boundaries (Abdelkader et al., 2022; Bhutto et al., 2020; Dejaeger et al., 2012). Each feature is assigned the same attention in the Naïve Bayes algorithm (Kamel et al., 2019; H. Zhang, 2004). These models are commonly used for classification that accepts discrete and continuous variables (Alhothali et al., 2022; Alsariera et al., 2022; Ha et al., 2020).

Over the years, DL has garnered extensive research. Several neural network architectures have been created for different data types. Recurrent neural networks such as long short-term memory (LSTM) and bidirectional LSTM (BiLSTM) are utilized mainly for text and speech data, while convolutional neural networks are employed for image processing. One variety of ANNs is known as a feedforward neural network. In this kind of network, the connections between the nodes do not create a loop. Feedforward neural networks get their name because information can only move in one direction, forward, inside the network, referred to as multi-layered networks (Kurani et al., 2023; Zupan, 1994).

The emotions used in this study are boring and mindfulness. The study used only two emotions, mindfulness and boredom, to simplify and control factors. Students may find choosing between two emotions easier, decreasing data ambiguity. We believe these two emotional states encapsulate vital dimensions of the online learning experience, providing valuable insights into online education’s potential pitfalls and positive aspects. They act as proxies for broader issues related to student satisfaction, engagement, and overall educational quality online. Altrabsheh et al. (2015) found that single-emotion models outperformed multi-emotion models concerning accuracy.

Multi-label classification generally consists of two approaches: problem transformation and adaptive algorithms. The first method converts multi-label data into binary classification data that can be solved with single-class algorithms. BR and CC are algorithms based on problem transformation. The adapted algorithm uses the whole label class rather than converting it to a single class. The algorithms MLkNN and BRkNN are these types of algorithms. The algorithms used in this work for multi-label emotion prediction are MLkNN (M.-L. Zhang & Zhou, 2007), BR (Luaces et al., 2012), BRkNN (Spyromitros et al., 2008), CC (Read et al., 2009), KNN, RF, and DT. The best parameter was estimated using the grid search method with a fivefold CV, and with the best estimator, the data were trained and tested with the stratified fivefold method. The Scikit-multi learn Python package is used for multi-label classification (Szymański & Kajdanowicz, 2019). The evaluation metrics are accuracy, F1-score, and hamming loss. Hamming loss analyses how accurately the classifier predicts each label (Wu & Zhou, 2017). The loss value varies between 0 and 1, and the lesser the value, the better the classifier.

Friedman statistical test was used to find the most efficient algorithm for all the ML models (Varzaneh et al., 2022). The rank assigned to each model helps to identify the most efficient model. The null hypothesis in the test is that there is no significant difference in the performance of the algorithms. The significant value is set as 0.05 in this study. The null hypothesis is accepted when the p value is greater than 0.05. The contribution of each predictor to the prediction is determined using SHAPley Additive Explanations (SHAP) (Lundberg & Lee, 2017).

3. Results

3.1. Data analysis

The survey collected 9194 responses from students studying in different colleges in GCC. We selected only GCC native students (9129 responses) and eliminated one empty cell response from this, as the students did not respond to the target labels. The final number of responses for the analysis was 9128 students. The demographic details are explained in Table B1. The highest response was from the Kingdom of Saudi Arabia (64.07%), and the least was from Qatar, where we received only one answer. Also, it can be visualized that most responses were with the synchronous online learning (SOL) method (65.99%). The asynchronous online learning (AOL) and blended online learning (BOL) methods have 8.01% and 26% responses, respectively. As per the response analysis, 93.89% of students belong to the public or governmental institutions type, and 58.5% were intermediate in computer use. Also, the table shows that 32% of students have had online learning experiences. 80.3% of the total participants were vaccinated against the coronavirus. Figure B1 shows the devices used by the students for online learning. Around 49% use laptops to learn during the pandemic, and fewer students opt for computers (10%).

Other than these questions, the students were asked about any behavioral changes (behavior-change) they felt during the learning. Fifty-eight percent agreed that they felt changes in their behavior during online learning. Some of the institutions provide e-libraries for learning purposes. How the students utilize the e-libraries are a concern. Nineteen percent of students mentioned that their institution does not offer e-books, while 51% used e-library resources. Thirty percent of students said that they do not use the option provided. Using the web camera properly is another concern for teachers. Fifty-eight percent of students preferred to turn off the web camera during learning. All these attributes, including the constructs, show 55 used as initial features or predictors for the ML models.

3.2. Classification model results

The BFS method selects relevant features according to the ordinal LR model, as the features contain categorical and ordinal values. The selected features for the satisfaction prediction model are Tech4, Teach11, Impact1, Impact3, Use1, Use2, and Use3. The emotion labeling model selects 36 features using the BFS method. The selected features are vaccinated, catch-covid, concern-covid, country, gender, grade, institution type, behavior change, computer experience, OL before, e-libraries, the device used, DL method, Int1, Int2, Int3, Int7, Exp3, Exp4, Exp5, Exp6, Exp9, Health1, Health2, Health3, Tech1, Tech3, Tech4, Teach2, Teach4, Teach5, Teach7, Teach8, Teach11, Impact2, and Use2.

3.2.1. Satisfaction model

The categorical features were converted into numeric values, and the data were cleaned and preprocessed before loading into the classification model. The fivefold CV was performed, and averaged the results for training and validation. The parameters of each model are explained in Table 2. The best-tuned parameters are max_depth = 6 for the DT classifier, the regularization (C) attribute = 2⁴, and gamma = 2⁻² for SVM with RBF kernel. The ensemble models RF and ET show the number of estimates, n_estimators = 310 for ET, and 400 for RF with maximum features as log₂. The tolerance rate (tol) is 0.01 for LR with C = 1e01, and the GNB smoothing variable is tuned to 1e–06. The KNN has two parameters: the number of neighbors, n_neighbors = 15, and p = 1 (implies Manhattan distance).

Table 2. Tuning parameters for each model in satisfaction classification.

Model	Tuning parameters	Range
DT	Max-depth	2 to 22
ET	N-estimator	10–400
	Max-features	Auto, sqrt, log2
RF	N-estimator	10–550
	Max-features	Auto, sqrt, log2
LR	C	1e–04 to 1e + 02
	tol	1e–01 to 1e–06
SVM-RBF	C	2^–1 to 2^5
	Gamma	2^–2 to 2^–7
KNN	N-neighbors	3–20
	p	1–4
GNB	smoothing	1e–01 to 1e–09
ANN	Hidden layers	1–5
	Learning rate	1e–01 to 1e–06

The ANN model is built with input, dense, dropout, and classification output dense layers. The hidden layer with four dense layers performs better than the other tuned hidden layers. Also, the learning rate is decreased every two epochs if the validation loss is stable or increasing. The total epoch is set to 50, and the EarlyStopping function is set to avoid model overfitting.

All eight ML models were trained on the dataset, showing a relatively small difference in the performance. Figure 3 shows the mean accuracy and SD according to the fivefold CV. It can be observed that the SD of the ANN results is comparatively larger than that of the SVM results. This discrepancy in SDs implies that the ANN results exhibit greater variability.

Figure 3. Training performance of ML models based on satisfaction classification. The error bars show the standard deviation. Training is performed by a fivefold cross-validation method.

The testing is performed on the remaining unseen data. Friedman’s test results and classification of test data are shown in Table 3. The Friedman test shows a significant value of 0.0002, less than 0.05; hence, we reject the null hypothesis. So, we concluded that there is a significant difference in the performance of the algorithms. According to the Friedman rank, the best algorithm is ANN for unseen data. The model shows 81% accuracy and 0.80 F1-score and precision values.

Table 3. Classification results of satisfaction prediction with feature selection with 30% unseen data.

Classifiers	Accuracy	Precision	Recall	F1-score	Friedman test rank (p value = 0.0002)
SVM	0.80	0.79	0.80	0.79	3
DT	0.80	0.79	0.80	0.79	3
KNN	0.80	0.79	0.80	0.79	3
ET	0.78	0.77	0.78	0.77	5.5
RF	0.78	0.77	0.78	0.77	5.5
LR	0.72	0.66	0.72	0.68	7.75
GNB	0.72	0.68	0.72	0.70	7.25
ANN	0.81	0.80	0.81	0.80	1

3.2.2. Emotion model

The multi-label classification consists of 36 features and two emotions: boredom and mindfulness. The students may feel bored or mindful, or even both, during their online learning. The data are preprocessed before loading to the model, and the parameters are tuned with preprocessed data. The tunned parameters are similar for DT, KNN, and RF, as shown in Table 2. The remaining model parameters are depicted in Table 4. The max-depth for DT is 4, the N-estimator for RF is 400 with log₂ features, and KNN has N-neighbors = 10. The MLkNN and BRkNN are based on the KNN model and have N-neighbors = 19 each. The BR and CC models are based on a NB estimator, where the alpha value is tuned as 1.

Table 4. Tuning parameters for each model in emotion classification.

Model	Tuning parameters	Range
MLkNN	N-neighbors	3–20
	smoothing	0.1–1.0
BRkNN	N-neighbors	3–20
BR	estimator-alpha	0.1–1.0
CC	estimator-alpha	0.1–1.0

The training data are evaluated on a fivefold CV, and the averaged results are saved. Figure 4 shows each model’s mean accuracy and SD based on emotion classification. It can be observed that the SD of the RF results is comparatively larger than that of the other ML model results. This discrepancy in SDs implies that the RF is a robust and adaptable ML model capable of addressing a wide range of problems. However, it is essential to be aware of its limitations, such as its potential for variability.

Figure 4. Training performance of ML models based on emotion classification. The error bars show the standard deviation. Training is performed by a fivefold cross-validation method.

Table 5 shows the result of multi-label classification with selected features on the unseen data. The Friedman test result shows a p value of 0.9196; hence, we accept the null hypothesis. So, there is no significant difference in the rank of model performance. Hence, we focused on hamming loss. The RF shows minimum hamming loss with 0.194 and has the highest accuracy and F1-score with 65.3% and 0.549, respectively.

Table 5. Classification results of emotion prediction with selected features with 30% unseen data.

Classifiers	Accuracy	Hamming loss	F 1-score	Friedman’s test rank (p value = 0.9196)
MLkNN	0.615	0.215	0.541	2.6
BR	0.591	0.227	0.538	5
BRkNN	0.608	0.217	0.464	4.6
CC	0.588	0.229	0.536	6
DT	0.621	0.210	0.516	3
KNN	0.607	0.217	0.439	5.6
RF	0.653	0.194	0.549	1

4. Discussion

The result of these ML models helps instructors to motivate and engage students more effectively. Because it yields reliable results, supervised learning was the most frequently used method to forecast students’ satisfaction and emotions. ML is applied to online learning in many studies. Most studies were related to the student’s performance (Alsariera et al., 2022; Kaddoura et al., 2022). Most studies split the dataset according to the train-test split method. In this work, a fivefold CV is performed for ML analysis, and then the model is tested on unseen data. The ANN model (81% accuracy) performs better than the other classifiers evaluated for satisfaction prediction. The rank of each model is shown in Table 3. The LR and GNB models showed the least accuracy, with 72% accuracy. We also tested the LSTM and BiLSTM models with one and four hidden layers on the satisfaction prediction data. The results showed that the LSTM model has an accuracy of 65% with four hidden layers, and BiLSTM has 67% with four hidden layers. So, all other ML models outperform the LSTM and BiLSTM models for our proposed study.

Multi-label classification consists of more than one dimension in the target class. Few articles exist for multi-label prediction with imbalanced data (Tarekegn et al., 2021). The RF classifier outperforms all other models in emotion (bored and mindfulness) prediction with an accuracy of 65.3%. The least performed model is CC, with 58.8% accuracy. The rank of each model is shown in Table 5.

Many research articles used ANN, a machine learning (ML) and data mining method, to predict student performance better. ANN is a strong pattern recognizer and a vital method for classifying various outcomes (Baashar et al., 2022; Kurani et al., 2023). RF is an ensemble method that combines many DTs and is suitable for multi-class and multi-label classification problems. The result varies according to different datasets, features, and classifier algorithms. RF classifier performs highly in student performance prediction studies (Lu et al., 2020). In another emotion study article (Altrabsheh et al., 2015), the complement NB classifier shows high accuracy, with 67% for excitement and other emotions and 0.47 F1-score for boredom, which deals with student tweets after the lectures. In comparison, the proposed study focused on the mindset of students during online learning and obtained better performance in terms of F1-score = 0.549. Also, Altrabsheh et al. (2015) found that single-emotion models outperformed multi-emotion models concerning accuracy. Table 6 compares the proposed study with the related literature studies.

Table 6. Comparison of the proposed study with related literature studies.

Reference	Outcome	Features	ML model	Result
Rezapour and Elmshaeuser (2022)	5-class	20	RF	72.58% accuracy
	3-class			83.87% accuracy
Ho et al. (2021)	7-class	16	Elastic net	65.2% explained variance
Lu et al. (2020)	3-class	16	RF	81.52% accuracy
Altrabsheh et al. (2015)	2-class emotion	–	CNB	0.47 F1-score
Proposed study	5-class	7	ANN	81% accuracy
	2-class multi-label	36	RF	0.55 F1-score

In the article (Ho et al., 2021), the feature importance parameter of ElasticNet regression is used to figure out the top features. The authors (Ho et al., 2021) mentioned that the top features were related to learning, instruction, and assessment constructs. Our study analyzes the essential predictors using the SHAP method. Figure 5 shows the essential predictors for students’ satisfaction with online learning prediction. From the total of 55 features, the BFS method selects seven features. The students prefer to use online learning if it is optional in the future curriculum (Use3), the existing functionalities of the online education platform match their learning needs (Use1), and Online learning is more attractive than traditional classroom learning (Use2) is the essential features. The remaining four predictors are Teach 11, Impact1 and 3, and Tech4 (as explained in Table A1). The SHAP plot shows how each feature helps in the contribution of prediction to class 0 to class 4 (strongly disagree to agree strongly).

Figure 5. Feature contribution of the satisfaction prediction model.

For multi-label emotion prediction, 36 features were selected based on the BFS method. The SHAP plot with the top 10 features for the multi-label prediction of emotion is shown in Figure 6. The feature value is shown in red and blue, where red indicates a high feature value, and blue indicates a low feature value. Based on this, we can conclude how the features contribute to each target prediction. The positive side of the SHAP value shows a positive prediction of boredom or mindfulness. The top features contributing to emotional boredom are concern-covid, country, Exp3, Exp4, Teach5, Health3, Teacg8, Teach11, Int1, and Use2. For example, Exp3 in bored emotion (Figure 6(a)) has red toward positive SHAP value. Students who agree that they face difficulty focusing on the computer all the time contribute more toward boredom (since it is coded as 0–4 from strongly disagree to agree strongly). Similarly, the highly contributed features for mindfulness emotion are computer experience, Int3, concern-covid, Health2, Impact2, Use2, DL method, Health1, gender, and behavior change. Here, the computer experience shows an important feature. The student who is an expert in computers feels mindfulness during online learning (the feature computer experience is coded as 0 for beginner, 1 for intermediate, and 2 for expert).

Figure 6. Feature contribution of the emotion prediction model. (a) Bored and (b) mindfulness.

In our study, the prediction model aims to identify potential learning challenges and areas where certain students may require more support. Once these students are identified, the instructor can employ various strategies. It might include providing more detailed and personalized feedback to these students, outlining specific areas for improvement, and suggesting targeted resources or exercises. Additionally, instructors might consider offering more one-on-one or small group tutoring sessions, allowing for tailored instruction. Instructors can also make adjustments to their teaching methods, perhaps incorporating more visual aids or interactive activities and gamification that might resonate better with those students. Continuous monitoring of these students’ progress and regular communication with them can aid in understanding their ongoing challenges and achievements, allowing the instructor to respond dynamically as needs change. Encouraging peer collaboration and study groups can foster a supportive learning community, helping those who may feel isolated or overwhelmed. The predictive analysis serves as a tool to guide these interventions, allowing the instructor to recognize potential challenges early in the learning process and take proactive measures to support those students who may be at risk. By addressing this in our study, we hope to contribute to a more inclusive and responsive personalized educational environment where all students are given the opportunities and support they need to succeed.

One of the strengths of this study is that it uses data from GCC students from different schools and colleges to implement the ML model. The GCC region, which has not previously been investigated, covers an area of one million square miles (2.6 million square meters) and houses approximately 60 million individuals with similar cultural backgrounds (World Population by Country, 2023). Also, different models like RF, DT, KNN, LR, GNB, ET, ANN, and SVM were used to predict the satisfaction model. Another important model is the multi-label classification model to predict students’ emotions during learning. MLkNN, BR, BRkNN, CC, KNN, DT, and RF are used for multi-label classification. A fivefold CV is performed on each model to avoid the overfitting problem, and unseen data are utilized to evaluate the model. Additionally, the features are chosen, and the dimensionality is decreased using the BFS approach.

4.1. Implications of the study

The study’s results show several important implications for educational institutions. The predicted models will help the instructors identify the less satisfied students and provide them with the necessary resources and motivation according to their needs. The emotion model provides valuable insights into students’ emotions during learning. It will help the institutions to focus on a more supportive learning environment. The association study between emotions and satisfaction levels highlights the need to provide comprehensive assistance for students. The findings suggest that online learning satisfaction was based on the usefulness, teaching style, and impact on learning. In contrast, the student’s emotion was related mainly to interaction, teaching style, health factors, and technical issues.

The relationships between student emotions and satisfaction levels are essential for our study’s goals and help us comprehend student well-being and learning experiences. The common features that help in predicting satisfaction and emotion are related to the teaching style (Teach11), online learning more attractive than traditional learning (Use2), and lack of knowledge in using tools for online learning (Tech4). The Chi-square test is evaluated on satisfaction and emotion, which resulted in no evidence to support that mindfulness and satisfaction are related (p value = 0.33). But, a significant relation exists between boredom and satisfaction (p value < 0.001). Students are more satisfied with their online education when they are satisfied with the teaching style and when they meet their learning needs. The difficulties that are thought to be involved in obtaining academic achievement (which includes satisfying teaching style and lack of knowledge in using online tools for learning) may contribute to developing negative emotions (boredom), leading to decreased satisfaction. Institutions might consider providing different teaching styles and engagement during the lectures to enhance students’ academic and learning experiences.

4.2. Limitations and future works

Even though the study has many strengths, there are a few limitations also. Investigating students’ learning styles for individual subjects is a future work that will help the instructor select different tools based on different subjects, as some subjects might require hands-on or experimental lab work that might not fit equally with other subjects. Another future enhancement can be studying the data with different feature selection methods and combining the results. Furthermore, our study opens the door to significant future exploration in two critical areas. First, investigating students’ learning styles across individual subjects presents an opportunity to tailor education to the unique demands of various disciplines. It includes recognizing the distinct needs of practical subjects requiring hands-on experiences, theoretical subjects demanding more abstract thinking, and creative subjects fostering personal expression. The selection of appropriate tools and methodologies for each subject can enhance the overall learning experience. Second, enhancing our data analysis through different feature selection methods offers exciting possibilities. Employing various statistical, ML, or hybrid approaches in feature selection may lead to a more nuanced understanding of the data. These two intertwined avenues of investigation hold the potential to significantly enrich both educational methodology and analytical precision, paving the way for more personalized and effective learning experiences.

5. Conclusions

Online education is one of the most significant components of modern learning, and it is likely to continue growing in the coming decades of education as well. In this study, we aimed to determine how satisfied students were with online learning. To do this, we used two ML classification models to identify their emotions and levels of satisfaction. Rather than diving into the nuances of specific subjects, we aimed to understand broader trends and attitudes, creating a more generalized understanding applicable across various educational contexts. These models provide valuable insights for institutions and instructors, enabling targeted guidance and support based on students’ emotions and satisfaction levels. Our exploration into different feature selection methods also offers opportunities to enhance the analytical robustness of our findings. By prioritizing a holistic view of online learning, our research contributes to human–computer interaction, highlighting the complex interplay between technology and human emotions. Our results show how important it is for online learning to be personalized and adaptable. They also pave the way for future research that could go deeper into subject-specific analysis or other complex parts of the online education scene.

Ethics statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Committee from the Office of the Vice Dean for Academic Affairs, Research and Graduate Studies, College of Engineering and Petroleum of Kuwait University (protocol code 24/2/533).

Consent form

Informed consent was obtained from all subjects involved in the study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data are not publicly available due to privacy reasons.

Correction Statement

This article was originally published with errors, which have now been corrected in the online version. Please see Correction (http://dx.doi.org/10.1080/10447318.2024.2326715)

Additional information

Notes on contributors

A. M. Mutawa

A. M. Mutawa received his PhD from Syracuse University, New York, USA, in 1999 in Artificial Intelligence and joined the Computer Engineering Department at Kuwait University (1999–present). He specializes in Artificial intelligence, and his research interests are Robotics, Expert Systems, Artificial Intelligence, Signal Processing, Pattern Recognition, Deep Learning, and e-Learning.

Sai Sruthi

Sai Sruthi received her MTech from SRM University in 2015 and her BTech from Calicut University, India, in 2012. She is currently a research assistant at the Computer Engineering Department at Kuwait University (2020–present). Her research interest includes E-Learning, Machine learning, Deep learning, and Natural language processing.

Appendix A

Table A1. The constructs and related questions of the study.

Constructs	Questions
Interaction	Int1: I interact with other students during class hours
	Int2: I interact with my teachers frequently during online learning
	Int3: There is no method to interact with the teachers outside the class
	Int4: The teacher’s interaction with students is good
	Int5: There is no proper parent–teacher interaction
	Int6: I have parental support at home for my studies
	Int7: My school/college provides me with the support needed during online learning
Perceived usefulness	Use1: The existing functionalities of the online education platform matched my learning needs throughout the COVID-19 pandemic
	Use2: Online learning is more attractive than traditional classroom learning
	Use3: I will still choose to use online learning if it is optional in future curriculum
Experience	Exp1: I get disconnected during live broadcasting, and this affects my comprehension
	Exp2: Sometimes, I cannot find a quiet place at home. So, I face a lack of concentration
	Exp3: I always face difficulty focusing on the computer all the time
	Exp4: I felt more isolated as a student
	Exp5: Heavy workload/homework compared to classroom learning
	Exp6: I do have disturbances from other students during online class hours
	Exp7: I get the chance to ask questions and discuss topics
	Exp8: I can achieve my targeted GPA during the online learning
	Exp9: Online learning is more challenging to adapt to than face-to-face learning
Health/social concern	Health1: My stress/anxiety level increased
	Health2: I have become lazier than before
	Health3: I often have health problems like headaches, eye strain, back pain
	Health4: I feel shy or nervous while attending online classes with the camera on
Technical problems	Tech1: Lack of gadgets for online learning in my home
	Tech2: I could not access all the materials of my course due to slow internet
	Tech3: No support is provided if any technical error occurs
	Tech4: Lack of knowledge of the use of online learning tools
Teaching style	Teach1: The course teaching method helped me understand the instructional content
	Teach2: I enjoyed the course teaching style
	Teach3: The teaching method was flexible and simple
	Teach4: The teaching method helped me improve my educational level
	Teach5: The teaching method encouraged me to study more
	Teach6: The teaching style met my expectations
	Teach7: The teaching did not include enough opportunities for discussion
	Teach8: The teaching method increased my desire to learn the subject
	Teach9: The teaching method used by the teacher-led to me achieving the course objectives well
	Teach10: The activities and examples were not sufficient to fully explain the educational content of the course
	Teach11: Overall, the teaching style was excellent
Impact of online learning	Impact1: I will not be able to return to school next term
	Impact2: My grades will be negatively affected
	Impact3: I am concerned about the time of completing or graduating the study
Concern-covid	How concerned are you about getting the coronavirus?
Satisfaction (target)	I am totally satisfied with the online learning method
Emotion (target)	The emotion you felt the most with the learning during the COVID19 pandemic. (bored, mindfulness)

Table A2. Internal reliability of the constructs.

Constructs	Cronbach’s alpha	Items
Interaction	0.723	7
Usefulness	0.924	3
Health	0.841	4
Experience	0.806	9
Technical	0.883	4
Teaching Style	0.908	11
Impact	0.774	3

Table A3. The selected features p value during the BFS method for satisfaction model.

Feature	Correlation coef.	SD error	p Value	97% confidence interval
				Lower bound	Upper bound
Tech4	−1.176	0.172	0.000	−1.512	−0.839
Teach11	−0.841	0.161	0.000	−1.157	−0.525
Impact1	1.234	0.205	0.000	0.832	1.636
Impact3	2.731	0.346	0.000	2.053	3.409
Use1	−0.406	0.166	0.015	−0.732	−0.080
Use2	−0.841	0.161	0.000	−1.157	−0.525
Use3	1.234	0.205	0.000	0.832	1.636

Table A4. The selected features p value during the BFS method for emotion model.

Feature	Correlation coef.	SD error	p Value	97% confidence interval
				Lower bound	Upper bound
Vaccinated	−0.313	0.072	0.000	−0.454	−0.172
Concern-Covid	−1.002	0.067	0.000	−1.134	−0.870
Catch-Covid	0.175	0.055	0.001	0.067	0.282
Country	−0.973	0.123	0.000	−1.213	−0.732
Gender	−0.154	0.050	0.002	−0.252	−0.055
Grade	0.968	0.250	0.000	0.478	1.458
Institution type	−0.432	0.121	0.000	−0.669	−0.196
Behavior-change	0.268	0.051	0.000	0.168	0.367
Device used	−0.499	0.097	0.000	−0.690	−0.309
OL before	−0.216	0.055	0.000	−0.324	−0.108
DL method	0.223	0.098	0.022	0.032	0.415
Computer-exp	0.489	0.078	0.000	0.335	0.643
E-library	0.152	0.067	0.024	0.020	0.284
Int1	−0.281	0.095	0.003	−0.467	−0.095
Int2	−0.272	0.104	0.009	−0.475	−0.068
Int3	−0.191	0.074	0.010	−0.335	−0.046
Int7	0.214	0.104	0.039	0.010	0.418
Exp3	0.280	0.106	0.008	0.072	0.488
Exp4	0.441	0.099	0.000	0.248	0.634
Exp5	0.194	0.084	0.021	0.030	0.359
Exp6	−0.541	0.087	0.000	−0.711	−0.371
Exp9	−0.299	0.113	0.008	−0.520	−0.078
Health1	−0.363	0.125	0.004	−0.608	−0.117
Health2	−0.501	0.126	0.000	−0.748	−0.253
Health3	0.372	0.095	0.000	0.186	0.558
Tech1	−0.250	0.099	0.011	−0.443	−0.057
Tech3	0.405	0.091	0.000	0.226	0.584
Tech4	−0.414	0.109	0.000	−0.627	−0.201
Teach2	−0.502	0.172	0.003	−0.838	−0.165
Teach4	0.419	0.151	0.006	0.122	0.716
Teach5	−0.373	0.141	0.008	−0.649	−0.097
Teach7	−0.203	0.084	0.016	−0.368	−0.038
Teach8	0.520	0.172	0.003	0.182	0.857
Teach11	−0.526	0.134	0.000	−0.788	−0.264
Impact2	−0.316	0.084	0.000	−0.480	−0.151
Use2	−0.296	0.079	0.000	−0.451	−0.141

Appendix B

Figure B1. The number of devices used by students for online learning.

Table B1. Demographic and other details of students.

Factor	Category	Total responses (N)	N (%)
Gender	Male	4248	46.54
	Female	4880	53.46
Grade	Graduate	601	6.58
	Undergraduate	8318	91.13
	High school	143	1.57
	Secondary school	47	0.51
	Primary school	19	0.21
Country	Kingdom of Saudi Arabia	5848	64.07
	Kuwait	2715	29.74
	Kingdom of Bahrain	547	5.99
	Sultanate of Oman	7	0.08
	United Arab Emirates	10	0.11
	Qatar	1	0.01
Institution	Public	8570	93.89
	Private	558	6.11
Distance learning method (DL method)	SOL	6024	65.99
	AOL	731	8.01
	BOL	2373	26.00
Computer Experience	Beginner	943	10.30
	Intermediate	5339	58.50
	Expert	2846	31.20
Previous Online learning experience (OL before)	No	6201	67.93
	Yes	2927	32.07
Catch corona once (catch-covid)	No	6668	73.05
	Yes	2460	26.95
Vaccinated against coronavirus	No	1797	19.69
	Yes	7331	80.31