Toward an artificial intelligence-based decision framework for developing adaptive e-learning systems to impact learners’ emotions

ABSTRACT

Learners’ emotional states might change during the learning process, and unpredictable variations of a person’s emotions raise the demand for regular assessment of feelings during learning. In this paper, an AI-based decision framework is proposed and implemented for e-learning systems that identify suitable micro-brake activities based on the learner’s emotional state through an evolutionary genetic algorithm to change learner’s mood and increase learning performance. This proposed framework was tested using a case study of English as a second language learner during one semester. The students were divided into two groups of participants (each group containing twenty students, forming a total of 40 students). The results of this study demonstrated the importance of learners’ emotions in their learning performance and proved the effectiveness of our proposed framework and the success of the recommended micro-break activities chosen based on learners’ emotions and preferences. The findings of this study have important practical implications in designing adaptive e-learning systems and learning management systems such as Moodle. They also contribute to theoretical implications in the field of AI and learner emotions by suggesting a novel approach to identifying, categorizing, and offering a learning path that can cater to the needs of individual learners.

KEYWORDS:

• Micro-break activities
• e-learning systems
• emotion regulation
• evolutionary algorithms
• genetic algorithm
• artificial intelligence

1. Introduction

There has been growing interest in the usage of computer-based, multimedia educational technologies such as Intelligent Tutoring Systems (ITSs) (Capuano et al., 2002; Albano et al., 2006) and Learning Management Systems (LMSs) (Darejeh et al., 2022b; Dagger et al., 2007). During learning with these systems, learners can experience various emotions and these emotions are critical in academic settings, as they can affect every aspect of cognition, and consequently, learning performance (Seli et al., 2016; Shen et al., 2009). Research has shown that anxious, angry, or depressed students cannot learn (Kort et al., 2001) and learners caught in these states do not absorb information efficiently or are not able to deal with new information well (Lee et al., 2011). Hence, emotional influences should be carefully considered in educational courses designed to maximize learners’ engagement and enhance their learning and long-term material retention (O’Regan, 2003).

Recently, the detection of emotional states in e-learning systems has been considered via various methods by many computer science researchers (Altuwairqi et al., 2021; Ashwin et al., 2020; Birjali et al., 2018; De Carolis et al., 2021; Ilić et al., 2021; Jithendran et al., 2020; Mukhopadhyay et al., 2020; Pise et al., 2020; Seli et al., 2016; Zakka & Vadapalli, 2019). There are also several prominent pieces of research on the learning preference and learner’s knowledge (Acampora et al., 2010; Martin & Carro, 2009); however, none of these studies has considered learners’ emotions during the learning process.

The existing approaches used in e-learning systems take the first step by detecting users’ emotional states and adapting learning contents according to this detection, but do not provide an e-learning environment that attempts to change learners’ moods, which can be considered as the leading cause of learners’ distractions (Lee et al., 2011). Here, the significant challenges are to determine: (a) the extent a dynamic e-learning system could help learners to improve their mood during the education process, after detecting the learners’ emotions; (b) whether adaptive e-learning systems can help learners to change their moods into optimal-for-learning emotional states by offering some micro-break activities; (c) the extent that these micro-break activities can be adapted to learners’ preferences, especially in vast e-learning networks involving hundreds of learners. In order to resolve this problem, we need suitable models and processes that can dynamically and intelligently create e-learning experiences which are adapted to learners’ emotions and preferences.

In this research, we aim to examine the impact of micro-break activities in adaptive e-learning systems on the learner’s mood and on their learning performance. We hypothesized that:

1. Adaptive e-learning systems that can recommend micro-break activities based on the learners’ preferences can increase learning performance.

2. The proposed framework using genetic algorithm can recommend appropriate break activities based on learners’ preferences.

3. Changing learners’ mood can increase learning performance.

In order to achieve this, we designed and implemented a framework to define a personalized e-learning experience based on learners reported emotional state. This framework suggests a sequence of micro-break activities according to the learner’s emotional state to maximize learning objectives during the learning journey. Since e-learning systems are web-based systems, we are facing an NP-hard problem that needs to optimize by artificial algorithms. Therefore, in our proposed framework we used the Genetic Algorithm in a hierarchical distribution scenario to derive the best match between different emotions and available micro-break activities in order to generate the most effective, personalized learning experience. The suggested framework is expected to be embedded in LMSs applications such as Moodle. It can help learners to change their moods to continue the learning process which could lead to an increase in learning performance in addition to an increase in learners’ engagement with the content (Tyng et al., 2017).

The next section discusses related works followed by the data collection and analysis procedure. Next, the results are presented, followed by a discussion and implications section. This paper ends with a brief conclusion.

2. Related works

In recent years, emotion and its role in the design and application of computing systems have gotten more popularity among researchers in the area of learning and teaching. Literature shows the importance of emotion in the teaching/learning process (Altuwairqi et al., 2021; Lee et al., 2011; O’Regan, 2003; Zulkifli et al., 2020). For example, in their study, Wang and Kang (2006) recommended a theory to make teaching more effective through working together with cognitive, emotive, and social factors Kort et al. (2001) developed a model of emotion related to the various phases of learning. They identified a range of emotional states that could be applied in computer-based systems, so that instructors could recognize the student’s emotional state during learning. They, however, did not go further than identifying the learner’s emotions.

In a similar study, Tyng et al. (2017) illustrated a basic evolutionary approach to understand the effects of emotion on learning and memory, and the functional roles played by various brain regions and their mutual interactions concerning emotional processing. They summarized the impact of emotion on memory and mapped its implications for educational settings. Furthermore, they used neuroimaging to prove the effect of emotional influences on learning and memory processes. According to their findings, consideration of emotional states might be helpful in the design of effective educational curricula and providing a conducive learning environment for both traditional “live” learning in classrooms and online learning. Just like Kort et al. (2001), they did not propose an approach that can help learners and teachers when the students are not in an optimal emotional mood.

On the other hand, some researchers have developed adaptive e-learning systems, of course, without considering learners’ emotions. These systems adapt the learning content based on different factors like learner location, time of the day, and preferences but not emotions. For instance, Martin and Carro (2009) proposed a new adaptive mobile learning environment and proposed a framework that provided different recommendations according to learner’s location and time. Also, Acampora et al. (2011) used ontologies and memetic agents in Web2 to propose a novel multi-agent e-learning system for defining a personalized e-learning experience. However, they merely focused on learners’ preferences.

More recently, some computer science researchers have considered the detection of emotional states in e-learning systems via various methods face and voice recognition. For example, Megahed and Mohammed (2020) used two intelligent systems, the so-called hybrid intelligent systems. They proposed an approach, namely, a loosely coupled integration between a convolutional neural network (CNN) and a fuzzy system. The CNN was used to detect a learner’s facial expressions and outperformed other CNN models on the same training benchmark. The fuzzy system was used to determine the next learning level based on the facial expression states extracted from the CNN and several response factors by the learner. In another study, Bahreini et al. (2016c) presented a validation study of their multimodal emotion recognition software system, developed and composed using existing software modules for real-time unimodal emotion analysis. They developed and implemented a software architecture framework called FILTWAM (Framework for Improving Learning through Webcams and Microphones). In this framework, they combined two emotion recognition software modules (using face and voice emotion recognition), described in their previous research studies (Bahreini et al., 2016a, 2016b). The aim of this research was to offer a real-time and multimodal solution to increase the accuracy of the combined face and voice emotion recognition software modules. Although these approaches might prove effective in experimental context, they are not very practical in education settings where face and voice recognition devices are not freely available. Therefore, this study attempts to address this problem by proposing and putting to test a framework that can be made accessible to learners and teachers on any device they have available.

Based on this brief review of the literature, it is clear that learners’ emotions can affect learning performance. However, none of the researchers has considered an easily accessible approach that can impact the learners’ emotional states in an e-learning environment. Therefore, in this work, we attempt to propose a framework for representing adapted micro-break activities based on the learners’ emotions and preferences.

2.1. Theoretical framework

This study is grounded in the cognitive theory of emotions. This theory has become “the touchstone of all philosophical theorizing about emotion, for or against” (Solomon, 2003, p. 1). From the perspective of this theory, emotion is understood as a mental state grounded on an appraisal of one’s experience that is “intelligent responses to the perception of value” (Nussbaum, 2001, p. 1). Therefore, emotions “involve thought of an object combined with thought of the object’s salience or importance; in that sense, they always involve appraisal or evaluation” (Nussbaum, 2001, p. 23). The cognitive theory of emotions also stresses the close connection between emotion and cognition and explains “the two equiprimordially constitutive manners of being-here” (Heidegger, 1996, p. 126) and acknowledges that emotions have “rich cognitive/intentional content” (Nussbaum, 2001, p. 11).

Cognitive theory of emotions suggests that emotions are key to the organization of cognitive processing and that understanding emotions by identifying their cognitive components and actions they induce is what makes education possible (Mortari, 2015; Nussbaum, 2001). This theory, therefore, has deemed suitable for the present study as it clearly indicates the relationship between emotions and education and the importance of understanding learner’s emotions while learning.

In addition to cognitive theory of emotions that framed the current work, this study also used the emotion ontology by Roberts et al. (2012) to categorize learner’s self-reported emotions and Kort’s emotion sets to determine the learning path devised for each individual. This study then used the knowledge model that was proposed and implemented by Acampora et al. (2011). This knowledge model enabled our system to extract appropriate micro-break activities based on the information it collected from the learners (i.e. their current emotions and the learner’s preferences in type of micro-break activities).

3. Methodology

3.1. The participants and the data analysis method

The participant of this study were forty teenage students (ranging from 10 to 15 years old learners, average = 12.3) enrolled in a general English course. These students came from 2 classes of 20 that were randomly selected from a pool of 17 classes studying at the language centre and they were randomly assigned to control and experimental groups. Each class had a mix of male and female students (see Table 1 for more detail). These participants were recruited based on the number of classes available to the researchers. Care was taken that the sample size does not negatively impact the reliability or validity of the results (more on this later in the section). The students of the experimental group (Group A) were taught using our proposed framework and they had access to break activities, while in the control group (Group B); the students were still taught using the same framework but they did not have access to break activities. These break activities were suggested to students by the system, based on their preferences and self-reported emotional state. The process of how the system selected activities will be explained in the next section.

Table 1. Age and gender distribution across control and experimental groups.

Age			10	11	12	13	14	15	Total
Access to break activity	Gender	Female	2	1	2	1	3	0	9
		Men	2	2	2	1	2	2	11
	Total								20
Lack of access to break activity	Gender	Female	2	1	0	3	2	2	10
		Men	1	2	4	1	2	0	10
	Total								20

As mentioned before, participants from both groups had access to our system. Students logged in the system and set their session settings, choosing their preferences and their emotions, they then accessed break activities. This way, before the teaching started, the system, using our proposed framework, tried to change their mood to an optimal-for-learning state through the activities that were created by the system. This was done by giving learners micro-break activities that work on changing their mood. Students from Group B logged in the system and set their session settings, choosing their preferences and their emotions, but they had no access to the micro-break activities and learned their lesson as usual.

The period of intervention was one semester and the learners’ performance was tested in terms of their ability in the learning of English before and after the intervention. The pre and post-test consisted of 40 multiple-choice questions, and 10 fill in the blank questions and were all of equal weight. An independent-samples t-test was deemed suitable for the purpose of this study and to compare the pre and post-test exam marks, for the control and experimental group. T-test is one of the most widely used statistical hypothesis tests (Yim et al., 2010) and is used to compare the means of the control and experimental groups when the experimental group is randomly assigned (Livingston, 2004). Since in this study, the participants were recruited based on random sampling, t-test is used. Although, a larger sample size can yield more reliable results, different scholars such as Gerald (2018) argue that a t-test returns reliable results even for a sample size smaller than 30 while some scholars believe that for reliable results, the number of participants could be as small as 2 (De Winter, 2013).

3.2. The architecture of our proposed framework

In accordance with hypothesis B, in this section, the proposed framework and its elements are presented. This framework has been executed through the emotions model of that suggested a model of the learning cycle and described a model of emotion including an axis, having negative valence to positive valence, that enabled them to formulate the learner’s emotional changes during the learning journey as a linear programming model (Figure 1). We grounded our research in this model and used different algorithms for executing a personalized e-learning experience that selected the most suitable activities for each learner and took into account the learner's personal characteristics, as well as preferences and their current emotional states.

Figure 1. Emotion sets possibilities relevant to learning (Adopted from Kort et al. (2001)).

As shown in Figure 2, the proposed framework consists of three main elements, namely, (a) cognitive factors, (b) setting of learners’ emotions, and (c) a recommender system. The details of each component of the framework are discussed in the following subsections.

Figure 2. The architecture of the proposed e-learning system.

3.2.1. Cognitive factors

Cognitive factors represent the knowledge reached by a learner at any given time. There are three significant elements in the sector of cognitive factors: the learners’ model, the model of knowledge, and the set target. The learners’ model is the leading actor in the whole learning process and defines the learners’ characteristics. In the stage of setting the target, learners determine their goals, and the model of knowledge describes the educational domain in the database of the e-learning system. It is used to teach the learners the learning concepts that match their aims. We will explain the three cognitive factors in the following subsections.

The learner’s model

The learner is anyone who connects to the learning system and selects target concepts (Acampora et al., 2010; Muehlenbrock, 2005). In our approach, the learner’s model shows the current knowledge of the learner, their emotional state during learning, and their preferences for break activities.

To get a better understanding of the learner’s current level of knowledge, we needed to assess the knowledge that is acquired by the learner during the learning journey and update the learner’s model accordingly. The assessment phases were used to evaluate learners’ knowledge of the studied subjects and were done by several kinds of tests such as multiple-choice or open-ended questions embedded in the e-learning system. Preferences of break activities for each emotional state were realized through a self-assessment phase using a questionnaire in the phase of registration. Break activities were then proposed by the system based on the different emotional states.

This study aims to change students’ moods through two groups of break activities. In the first group, there is a semantic point between any break activity and learning content (Group A). For example, in our case study, the framework suggests some English games when a learner selects to play serious games as a break activity. Another example of this group is when learners choose to watch a short video from a pool of short videos related to the teaching content. The second group of micro-break activities, aim to create a calm and relaxed state of mind for the learners. The learners can personally choose the break activities without semantic relation. For example, they can do some short meditation or chat with their friends when anxious or worried (Group B). For a full list of micro-break activities, please refer to Table 2.

Table 2. Property values in relaxation activity preferences.

Property	Vocabulary
Micro-break activity	Playing serious video games related to the teaching content, viewing summary diagrams of the teaching content, reading more details about the teaching content, some short physical activities, meditation activities, doing problem-solving exercises, reading examples, watching a short video, chatting with friends
Interactivity type	Physical, mental, mixed, undefined
Interactivity level	Very low, low, medium, high, very high
Typical activity time	Depends on the activity type
Emotional state	Anxiety, worry, discomfort, comfort, hopeful, confident-ennui, boredom, indifference, interest, curiosity, intrigue, frustration, puzzlement, confusion, insight, enlightenment, epiphany, dispirited, disappointed, dissatisfied, satisfied, thrilled, enthusiastic-terror, dread, apprehension, calm, anticipatory, excited
Age	Child, teenager, adult, elderly, old

The break activities are represented by a list of property names and property values (Acampora et al., 2010). These properties place in a set consisting of the elements, including break activity, interactivity type, interactivity level, typical activity time, emotion state, and age. Table 2 shows how these properties were identified in the present study by presenting sample values that are relevant for each property to demonstrate.

After determining the learner’s knowledge level, emotional state, and preferences for the break activities, a suitable exercise based on their preferences was suggested if they were in a non-optimal emotional state for learning.

As it is demonstrated in Table 2, the properties of learner preferences for a break activity should be a combination of different items. For instance, we can define a break activity preference of a sample learner as follows:

{Watching a short video, Mental, Medium, 8 minutes, Boredom, Home, Teenager}

Therefore, it can be stated that the learner is a teenager who likes to watch a short video that is mental for minutes at home when he/she feels bored.

The model of knowledge

The model of knowledge describes the educational domain in the database of the e-learning system used to teach learning concepts to the learner. The mechanism used by the Model of Knowledge is called ontology (Gruber, 1995; Guarino, 1998), which is defined as a formulated solution that shows the relationship between different pieces of learning concepts, and helps to propose an appropriate learning concept by considering pre-requirements and learner’s knowledge. The knowledge model presented used in this study is adopted from Acampora et al. (2010). This knowledge model enabled our system to identify appropriate micro-break activities based on the information it collected from the learners about their emotions and preferences.

The set learning path

The learner chooses the target concepts they intend to learn, and the system uses the Learning Path Generation Algorithm proposed adopted from Acampora et al. (2011) to define the learning path for each learner. This algorithm extracts appropriate teaching concepts from the database of the model of knowledge based on the target concepts selected by the learner. Since the focus of the current study is on learners’ emotions, not the learning contents, the details of this algorithm will not be covered in this paper.

Setting the learner’s emotion

In order to get the artificial algorithms to suggest the most suitable break activities to learners, we ask the learners, to log in to the system, and to choose their moods through a dropdown menu (see Figure 9). After learners choose their emotions, the emotion ontology represented by Roberts et al. (2012) is used to categorize the emotions. As it could be seen in Figure 3, if the learner has an emotional state such as anger, disgust or sadness, then “the learner’s emotion estimate module” of our proposed framework sets the status to negative, and the system recommends the most suitable break activities to return the learner to an suitable for learning emotional state. If, on the other hand, the learner has a state of emotion such as joy or love, then “the learner’s emotion estimate module” sets the state to positive. Then, the system starts the next step of learning, and the module recommends the most suitable activity to continue learning.

Figure 3. Emotional ontology of the six Ekman emotions (plus LOVE) (Roberts et al., 2012). Solid lines indicate inheritance, and dashed lines indicate the opposition.

In this emotional ontology, learners need to specify meaningful relationships and flows by sorting the emotional states that are called Emotion Paths (EPs). In an EP, after detecting the emotional state of the learner, emotion is placed on the Kort’s Axis (Figure 1) to lead the learner to a suitable emotional state. Based on the Kort et al. (2001) model, learners commonly experience emotional variations during the learning journey, from frustration to excitement, from boredom to fascination, which can cause a positive or negative educational experience.

The recommender system

Based on the information the system collects from the learners, the recommender system automatically suggests a series of break activities adapted to the learner’s preferences and current emotional state. For instance, if the system recognizes that the learner is not in a good mood, the recommender system suggests some break activities to the learner to take them to an optimal-for-learning mood. This is done by using a genetic algorithm implemented in the system. Before going deeply into the details of the algorithms related to the personalized e-learning system, we illustrate the way the break Activity Object (AO) is related to the learner’s emotion within the framework.

The main idea of the activity object is to break the activity content down into small chunks that can be reused in various learning conditions. In order to allow the reusing of the activity objects, they have to be annotated with the metadata. Figure 4 shows the semantic link between the activity objects and emotions in the emotion ontology.

Figure 4. The semantic link between activity objects and emotion ontologies.

As can be seen in Figure 4, the same activity object may control more than one emotional state within the corresponding ontology. For example, the activity object AO3 controls the states of Worry and Anxiety. Furthermore, it is possible to have more than one activity object controlling the same emotional state. This is the case for AO4 and AO6, both of which cover the state of Comfort. Also, this is true for AO2 and AO3, both of which cover the states of Anxiety and Worry.

In order to denote the relation between an activity object and an emotional state, we use the format (AO, Emotion X), which states that the activity object AO controls the emotion X. For example, Figure 4 shows metadata of the break activity objects that are available in the Activity Object Pool, and Table 3 reports cognitive states and break activity preferences of a sample learner and the way anxiety converts to confidence. In this example, we assume that the learner chooses the Anxiety mode. So, our system determines that anxiety should be changed to confidence. Then, based on the Kort’s model, the Learner’s Emotional Estimate System (LEES) extracts the following personalized emotion path (Table 4): $\mathrm{E}\mathrm{P}=\left[\mathrm{A}\mathrm{n}\mathrm{x}\mathrm{i}\mathrm{e}\mathrm{t}\mathrm{y},\mathrm{W}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{y},\mathrm{C}\mathrm{o}\mathrm{m}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{t},\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{f}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}\right]$The Recommender System (RS) performs a binding between available activity objects and emotional states in LEES. In particular, binding tries to minimize the number of activity objects. It selects the activity objects whose metadata best match the learner’s preferences, which in this example are $\mathrm{R}\mathrm{S}=\left[\mathrm{A}\mathrm{O}3,\mathrm{A}\mathrm{O}5,\mathrm{A}\mathrm{O}6\right]$ (see Table 4). In order to check if the learner reached an optimal emotional state, the system executes an assessment phase and saves the results to the database. The learner’s emotional state is updated based on the latest assessment results. The learning process continues based on the improvement of the learner’s emotional state.

Table 3. A list of sample states and activity objects.

AO1 (Doing problem-solving exercises, Mixed, Medium, 10 min, Anxiety, Teenager)

AO2 (Chatting with a friend, Mental, Medium, 5 min, Anxiety, Teenager)

AO3 (Playing serious games, Mixed, Medium, 5 min, Worry or Anxiety, Teenager)

AO4 (Viewing a summary diagram, Mental, Medium, 8 min, Comfort, Teenager)

AO5 (Reading an example, Mixed, High, 6 min, Confident, Teenager)

AO6 (Watching short videos, Mental, Medium, 5 min, Comfort, Teenager)

Table 4. Cognitive states and micro-break activity preferences for a sample learner.

Emotional state: Anxiety, worry, comfort, confident
Micro-break activity preferences: Playing serious games, reading examples, watching short videos Interactivity type: Mental Interactivity level: Medium Typical activity time: 15 min Age: Teenager

The algorithm of the recommender system

In this study, the evolutionary Genetic Algorithm is used to optimize the learner’s emotional state during the learning journey in a linear formula. The input of the algorithm is a collection that includes all the break activities for each emotional state. For each activity, $\mathit{t}$ and $\mathit{r}$ denote runtime and efficiency, respectively. Runtime is the duration of each break activity, and efficiency is the percentage of the proposed break activities match user preferences. (1) $\mathrm{\forall }{a}_i\in A,a_i=〈t_i,r_i〉.$(1) As explained in Section “The recommender system”, break activities are located in different emotional collections, including positive, negative, and neutral. For example, the positive collection has 15 emotional states, each of which has some break activities. As can be seen in what follows, the collection involves emotional states, each of which including several break activities: (2) $C=\{c_1,\dots ,c_n\},c_i=\{a_{i1},\dots ,a_{im}\}.$(2) Each solution created by the algorithm is a structured array of $x$ whose elements are sorted based on the emotional categories from negative to positive: (3) $x=\{x_1,\dots ,x_n\},x_j\in c.$(3) For each $x_j$ that belongs to $x$, there should be at least one ci belonging to C, such that the total performance of x is maximized in ci, and their total time is less than the total time of the array of $x$ for any group. The linear programming model which formulated the whole process is described as follows. (4) $\begin{array}{rl}& max\left(\mathrm{A}\mathrm{g}{\mathrm{e}}_{\mathrm{F}\mathrm{l}\mathrm{a}\mathrm{g}}\right(\sum _{i=1}^m\sum _{j=1}^n{x}_j{r}_{ij}+{\displaystyle \frac{1}{{\sum }_{i-1}^n{\sum }_{j-1}^m{x}_j{t}_{ij}}\left)\right)}\\ & \sum _{j-1}^m{x}_j=1x_j\in \left\{-1,1\right\}\\ & r_{ij}\in Ni=1,\dots ,mj=1,\dots ,n\\ & t_{ij}\in Ni=1,\dots ,mj=1,\dots ,n\\ & \mathrm{A}\mathrm{g}{\mathrm{e}}_{\mathrm{F}\mathrm{l}\mathrm{a}\mathrm{g}}=1\mathrm{A}\mathrm{g}{\mathrm{e}}_{\mathrm{F}\mathrm{l}\mathrm{a}\mathrm{g}}\in \{\begin{array}{c}1\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{o}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l}\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}\mathrm{e}\mathrm{r},\\ -1\mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}.\end{array}\end{array}$(4) The constraints should be satisfied to guarantee that each emotional state will be exactly covered by only one activity object. The total time of each activity is not more than the activity time announced by the learner. $\begin{array}{rl}& \sum _{j-1}^m{x}_j=1x_j\in \left\{-1,1\right\},\\ & t_{ij}\le \mathrm{s}\mathrm{u}\mathrm{m}\left(\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{o}\mathrm{f}\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}{\mathrm{y}}_{ij}\right)i=1,\dots ,mj=1,\dots n.\end{array}$The linear formula explained in this topic will be optimized using the Genetic Algorithm. The following section will describe the algorithm in detail.

The genetic algorithm

In order to find the best break activities among different activities, it is necessary to set the parameters of the Genetic Algorithm in the evolutionary schemas. For this to happen, we need to define

1. the way potential solutions for the problems should be represented, that is, a chromosome;

2. the fitness function to indicate how good the solutions computed by the algorithm are;

3. the genetic operators, including crossover and mutation that are used to run the algorithm.

In that case, chromosome L will be defined as a numerical vector composed of N components (genes) and should be between [1 − M]. Suppose that we assume $M$ as some emotional state and $N$ as the number of break activity objects. In this case, chromosome $L$ will be defined as a numerical vector composed of $N$ components (genes) and should be between $\left[1-M\right]$. Assuming that the $i\mathrm{t}\mathrm{h}$ component is equal to $j$ ($L_i=j$), the jth activity object covers the ith emotional state. Figure 5 shows a sample chromosome when $M=7\mathrm{a}\mathrm{n}\mathrm{d}N=10$.

Figure 5. A sample chromosome.

In a genetic algorithm, the fitness function explains the suitability of solutions (Tang et al., 2007). The fitness function of our algorithm is (5) $\mathrm{f}\mathrm{i}\mathrm{t}\mathrm{n}\mathrm{e}\mathrm{s}\mathrm{s}\left(L\right)=\mathrm{A}\mathrm{g}{\mathrm{e}}_{\mathrm{F}\mathrm{l}\mathrm{a}\mathrm{g}}\left(\sum _{i=1}^m\sum _{j=1}^n{x}_j{r}_{ij}+{\displaystyle \frac{1}{{\sum }_{i-1}^n{\sum }_{j-1}^m{x}_j{t}_{ij}}}\right).$(5) As can be seen in Equation (5), $\mathrm{A}\mathrm{g}{\mathrm{e}}_{\mathrm{F}\mathrm{l}\mathrm{a}\mathrm{g}}$ will be equal to 1 if the activity is proportional to the learner’s age; otherwise, it will be equal to −1. Also, $x_j$ is an array with two values for each category. If the jth object is selected from the ith category in the emotional path, its value will be 1. Otherwise, it will be −1. In this equation, $r_{ij}$ is the performance and $t_{ij}$ is the running time of the jth object of the ith category. After defining the chromosome template, it is possible to introduce the crossover and mutation operators extracted by each core to evolve its subpopulation portion. Our proposed genetic algorithm uses a typical one-point crossover and mutation to create the next generation of a genetic algorithm using the elitism method. Figures 6 and 7 show the crossover and mutation samples.

Figure 6. The crossover operator.

Figure 7. The mutation operator.

Elitism means that after crossover and mutation, among the existing chromosomes, the chromosomes with the best fitness will be selected for the next generation.

In this case, the best chromosome is the chromosome whose fitness has the maximum value. In order to find the probabilities of crossover and mutation, the trial-and-error method was used. The best crossover probability was 0.95, and the mutation probability was 0.05.

3.3. Case design

According to hypotheses A and C, the proposed framework is implemented in an e-learning system in order to find the effect of the algorithm that proposes appropriate micro-break activities and increasing learning performance. First of all, the participants register in our system and, at the same time, choose their preferences from the existing form according to Table 2 (see Figure 8). When the learners log in to the system, they are first asked about their mood through a dropdown menu (see Figure 9).

Figure 8. The page containing the selection of break activities.

Figure 9. The login page.

The system sets a session of learners’ emotions according to their selections. When the learners do their break activities, their session moods will be updated to the new moods. In this case, we consider emotions placed on the first row at the Kort axis (see Figure 1) and help the participants to change their mood from anxiety to confidence (see Figure 10).

Figure 10. The suggestion page.

The genetic algorithm embedded in the system is implemented in the background and suggests the most suitable list of break activities through learners’ preference and age. In the final step, the system provides features for the learner to do these activities according to the result of the genetic algorithm.

The following section explains the results of the experiment in detail.

4. Results

The descriptive analysis of the two groups in the pre-test revealed (M [Mean] = 10.22, SD [standard deviation] = 2.52) for the control group and (M = 10.8, SD = 3) for the experimental group. In the post-test, these values were (M = 14.87, SD = 1.54) for the control group and (M = 16.75, SD = 1.92) for the experimental group (see Table 5).

Table 5. Descriptive statistics of the control vs experimental groups.

Access to break activities		N	Mean	Std. deviation	Std. error mean
Pre-test	Lack of access to Break Activities	20	10.2250	2.52475	0.56455
	Access to Break Activities	20	10.8000	3.00394	0.67170
Post-test	Lack of access to Break Activities	20	14.8750	1.54643	0.34579
	Access to Break Activities	20	16.7500	1.92969	0.43149

To analyse the collected data, an independent-samples t-test was conducted to compare the exam marks collected from our two groups of participants (i.e. control and experimental) before and after the start of learning. The result showed no significant difference in the pre-test between students in the control group and those in the experimental group t(38) = 0,655, p > .05. However, the results showed a significant difference in the post-test scores of the control group and the experimental under the same conditions t(38) = 3,391, p < .05 (see Table 6). To ensure that the homogeneity of variances between the control and the experimental group, the Confidence Interval of the Difference was calculated which showed a 95% confidence level with lower limit of −1.2 and upper limit of 2.3 for the pre-test and lower limit of 0.75 and upper limit of 2.9 for the post-test.

Table 6. The independent-samples test of the control vs experimental groups.

Experiments	F	t	df	Sig	Mean difference	Std. error difference	95% Confidence interval of the difference
							Lower	Upper
Pre-test	0.440	0.655	38	0.516	0.57500	0.87744	−1.20128	2.35128
Post-test	2.078	3.391	38	0.002	1.87500	0.55295	0.75561	2.99439

Figure 11 illustrates the distribution of exam marks for the control and experimental group in pre and post-test conditions. In this diagram, boxplots demonstrate that interquartile ranges were typically between 9 and 12.5 for both groups in the pre-test. As illustrated in Figure 11, the lower quartile stood about 9, the mediums were near 10, and finally, the upper quartiles were approximately 12 for pre-tests under both conditions. In contrast, the results showed a significant difference between the control and experimental groups in the post-test. For students in the experimental group, i.e. with access to break activities, the lower quartile was roughly 15, the medium stood at 16, and the upper quartile was just over 18. Nonetheless, when students had no access to break activities, the lower quartile, the medium, and the upper quartile were 14, just over 15, and 16, respectively. In addition, the maximum mark of post-test exams stood about 20, and the minimum was 14 with access to micro-break activities. At the same time, without any access to break activities, we had a maximum of 18 and a minimum of 12.

Figure 11. Boxplot results.

As it is explained in the methodology, the data are collected throughout one term. Figure 12 shows a summary of the type of break activities that students selected as their preferences, during 16 sessions of the term (i.e. students in the experimental group).

Figure 12. Break-Activities * Session Crosstabulation.

Figure 12 demonstrates the micro-break activities suggested by our proposed algorithm across all sessions.

Figure 13 shows that playing a game was the most popular micro-break activity at 31.25%. The second most significant percentage of suggested chosen micro-break activities came from watching a short video at about 14%. Chatting with friends was the third-largest micro-break activity at 12.5%. Meditation activity was the most negligible break activity at about 1%.

Figure 13. Break-Activities.

As shown in Figure 14, boredom accounted for the most significant proportion of emotional mood during learning by just under 38%. The second-largest percentage of learners in unsuitable moods came from worry, at nearly 17%. Puzzlement was the third-largest percentage of the emotional state at 5%.

Figure 14. Emotion.

5. Discussion

The results of the present study suggest that the recommendation of break activities according to students’ preferences significantly improves their learning performance. Specifically, our results suggest that completing break activities suggested by the system can change the learners’ mood from a non-optimal mood for learning to a more optimal mood for learning and hence increase their learning ability. In this section, we delve deeper into these results and examine and compare the existing literature.

The results of the present study highlight that the learner’s emotional state is an important factor in learning and learner performance as is also concluded by O’Regan (2003). This link between emotion and learning has been emphasized by several other studies that have found a vital connection between emotion and learning, cognitive processes, memory, and decision-making (Clore & Gasper, 2000; Ferro, 1993; Hascher, 2010; O’Regan, 2003; Weiss, 2000; Worthman, 1999; Zulkifli et al., 2020). The results demonstrate that students who had access to e-learning system with micro-break activities had a better learning performance than students in the control group with no access to the micro-break activities. Therefore, hypothesis (A) which assumed that adaptive e-learning systems that can recommend micro-break activities based on the learners’ preferences can increase learning performance is supported. A study by Altuwairqi et al. (2021) suggests that this better learning performance could be because of higher engagement levels of students when they are in a more optimal emotional state.

The results also highlight that emotions that learners experience are not fixed and that individuals can change their moods by getting help from tools in their environment (in this case the micro-break activities our system recommended to them). The fact that emotions are not fixed and can be changed is well aligned with the literature on emotion (Danvers, 2016; Jacobs & Gross, 2014; Li & Wang, 2018; Zembylas, 2020). The ability of our system to improve learner’s mood is proof that our framework that utilizes genetic algorithm was successful in identifying the learners’ emotions, identifying an effective path that leads students to a more optimal emotional state for learning, and deploy the process by taking the learners through this path. These findings support hypothesis (B) which assumed that the proposed framework using genetic algorithm can recommend appropriate break activities based on learners’ preferences.

Another important piece of finding is that this study demonstrated that micro-break activities can be used as effective tools to increase learning performance. Therefore, hypothesis (C) which assumed that changing learners’ mood can increase learning performance is supported. Micro-break activities could be extremely useful in both face-to-face and online teaching and when the learners’ emotional state is not optimal for learning. They can be used to make the learners’ emotions more optimal and hence increase their learning performance. These activities could be used irrespective of the subject matter being taught; however, they might be more beneficial when teaching challenging tasks (tasks with high interactivity elements) that require more cognitive resources since they can impact learners’ cognitive load and change their mood and decrease learning performance (Darejeh et al., 2021, 2022a) .

Studies on micro-break activities are mainly in the area of workplace and wellbeing and there is a lack of research on micro-break activities in the area of education and only a handful of studies have examined the impact of micro-break activities. One such study on micro-flip activities was done by Fidalgo-Blanco et al. (2017) who found that micro-flip activities have a direct positive impact on student learning and that this effect is by no means subject-dependent. Studies in the area of workplace and wellbeing have also found similar results and can vouch for the effectiveness of micro-break activities (see for example, Kim et al., 2022; Kim et al., 2017; Nie et al., 2021).

From a computational perspective, we proposed a linear formula that recommends an array that includes the best sequence of micro-break activities for learners according to their preferences. This study tried to optimize a linear formula through the genetic algorithm (Acampora et al., 2010) mentioned in section “The genetic algorithm”. The results proved that the genetic algorithm is a promising algorithm in order to recommend appropriate micro-break activities based on learners’ preferences. These results are also supported by several studies in the literature (e.g. Dwivedi et al., 2018; Hssina & Erritali, 2019; Kratica et al., 2001; Vanitha et al., 2019). The proposed algorithm can be integrated in different types of adaptive e-learning systems or learning management system such as Moodle as a way to optimize learners’ mood when learning different subject matters.

6. Conclusion and ideas for further research

Focusing on learners’ emotions while learning as an essential aspect of learning, this study examined the impact of micro-break activities on the learner’s mood and on their learning performance. We hypothesized that our proposed framework, which is grounded in the cognitive theory of emotions, will be able to recommend micro-break activities that are able to change the learner’s mood to make them ready for learning and in turn, improve their learning performance. Based on our proposed framework, utilizing the emotion ontology by Roberts et al. (2012), Genetic Algorithm, and Kort’s emotion sets to respectively categorize learner’s self-reported emotions and determine the learning path devised for each individual, the system suggested adaptive break activities to learners, based on their preferences to return them to an optimal learning condition.

In the first step, the learner’s emotion was estimated by asking the learners to choose their current emotions from a dropdown list at the time of login. In cases that the learner had an non-optimal emotional state, an emotional path based on the Kort’s model was generated, including different micro-break activities according to the learner’s preferences. The Genetic Algorithm was used to adapt each micro-break activity to the related emotional state. The results of this study showed that our proposed framework was successful in changing learner emotions after taking the learners through the micro-break activities and in as a result improving the learners’ performance in learning the content matter. This points to the important role emotions play in the learning process showing that when learners are in their optimal emotional status, they are able to learn better. The results also point to the potential of micro-break activities and how they could be used to change the tone of teaching and learner’s emotions.

The findings of this study could contribute to and facilitate online learning experience for both the learners and teachers. These findings can help learners and teachers of different subject areas by offering a framework that suggest suitable micro-break activities that, based on the results of this study, are able to change learner’s emotional state to a suitable state for learning. This will in turn create a more effective environment for learning. The fact that micro-break activities proved effective in this context is of itself interesting and can count as a big contribution given that these activities can change the mood of learners, and in turn raise their learning performance, in a short amount of time. These findings also have theoretical implications for researchers in the field of AI and learner emotions by suggesting a novel approach to identifying, categorizing and offering a learning path that can cater to the needs of individual learners.

Although the results of this study proved promising, we need to keep in mind that since this system, using our proposed framework was tested on a small group of learners, it might not have fully uncover all the possible emotional states learners can experience. A bigger sample size is also able to strengthen the results of the statistical analysis. It is therefore essential to keep an open mind and investigate other possible emotional states that might require a different approach and aim for a bigger sample size in further research on this topic. Additionally, collecting qualitative data through interview or focus group could have widened our perspective as to how the learners feel about the process and the micro-break activities. Another factor that might have played a role in the results, is the subject matter (i.e. English) taught to the participants of this study. Since learning English requires substantial amount of interaction, emotions might have played a more significant role. It is, therefore, important that this framework is examined in other subject areas that rely on more individual learning such as math and science courses. Since we face the NP-hard problem that needs to be optimized in an effective way, the findings of this study could also be enriched by studies that compare our proposed framework with other available artificial intelligence algorithms. Examining these factors in future research could contribute to better understanding of the relationship between emotions and e-learning systems and identifying ways of making the artificial intelligence algorithms more effective. Therefore, in future work, there is an opportunity to find the most optimized artificial intelligence algorithm that aims to optimize learner’s mood in a timely manner.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Tayebeh Sargazi Moghadam

Tayebeh Sargazi Moghadam holds a Master’s Degree in Computer Science and Engineering with an honour’s degree (A) from The University of Sistan and Baluchestan, which is a well-known University in southeast Iran. Her master thesis was about artificial intelligence algorithms in E-Learning systems. She was employed at the University of Zabol as a computer engineer by the Information Technology department in 2008. Tayebeh was awarded by “the Research and Technology department” and “the Information and Technology department” as the Best Programmer for preparing more than ten internal projects at this university. She moved from Zabol university to the University of Sistan and Baluchestan in 2017. Since then, she has worked as a program analyst and PHP web developer at the University of Sistan and Baluchestan. Her research interest includes machine learning, artificial intelligence, cognitive and computational psychology, intelligent tutorials, human-machine interaction, social tutorials network, artificial intelligence in digital finance, and evolutionary algorithms.

Ali Darejeh

Ali Darejeh has a PhD in Computer Science and Engineering from the University of New South Wales (UNSW) and he is currently working as a lecturer and computer science researcher in the field of Human Computer Interaction at UNSW. He has over 10 years of teaching, research and industry experience and frequently publishes in high-rank journals. His major research interests are educational technologies including virtual/augmented reality, brain computer interfaces, serious games, gamification, e-learning systems, and user interface design for users with limited computer literacy. Ali conducts interdisciplinary research focusing on integrating new technologies and techniques such as augmented reality, virtual reality, and gamification with psychological and educational theories such as cognitive load, self-determination, and constructivism in order to design software for people with special needs, and e-learning platforms that can increase users’ motivation and learning performance. Other than academic degrees and research experiences, Ali has years of industry experience as a UX researcher and gamification designer in different industries such as oil and gas, medical, finance, and insurance. In addition, he is the founder of Research Tech Software Solution and Degree game studio, that focus on developing game-based learning platforms, and innovative mobile apps.

Mansoureh Delaramifar

Mansoureh Delaramifar has a PhD in linguistics from the University of Sistan and Baluchestan. She is currently working as a lecturer in general linguistics at the University of Sistan and Baluchestan. She has over 10 years of teaching and research experience and frequently publishes in high-rank Iranian journals. Her primary research interests are computational linguistics, frame semantics, phonology, and syntax. Mansoureh conducts interdisciplinary research focusing on the frame structure of metaphors in COVID-19 news and other social dimensions. Other than academic degrees and research experiences, Mansoureh has years of educational experience as an English instructor for teenagers.

Sara Mashayekh

Sara Mashayekh is a lecturer and researcher in the School of Education, the University of New South Wales. She has over 18 years of experience in teaching and research and has been actively involved in research projects at university and state levels to design tools to enhance learning experience and teaching effectiveness. Her research interests are educational technology, teacher education, learner agency, and pedagogical tasks.