Mental workload prediction model based on information entropy

Abstract

This paper introduces the concept of information entropy in studying mental workloads to predict the mental workload of an urban railway dispatcher and thereby ensure safe rail system operation. This study combines factors that can influence mental workload, including visual behaviors required for dispatchers to obtain information, information display duration, and the amount of information in order to establish a comprehensive mental workload prediction model. Experimental monitoring tasks were carried out on a simulation dispatch interface platform to verify the model’s validity. Three assessment methods (task performance assessment, subjective assessment, and physiological assessment) were adopted to measure the mental workload levels of dispatchers under different task conditions. The results demonstrate that the model’s theoretical prediction value significantly correlates with the various experimental results, thereby verifying the model validity and indicating that it can be used to predict the mental workload for different dispatch tasks, to provide a reference for work performance evaluation, and in designing optimized dispatch display interfaces.

Keywords:

• Information entropy
• mental workload
• amount of information
• ergonomics

1. Introduction

The urban public railway dispatcher who supervises and monitors railway operations performs a key function in ensuring railway safety. The increasing automation of railway systems has transformed the work of the railway dispatcher from manual operation to monitoring and troubleshooting. While this transition has dramatically reduced railway dispatchers’ workloads, it has also significantly increased the amount of information that dispatchers must monitor. When an emergency occurs, a large amount of information, such as graphs and text appears on the dispatcher’s interface, and the dispatcher must determine the specific information that is relevant to the system’s safety. However, human mental resources are limited,[1] and overwhelming amounts of information create a high mental workload that can interfere with the dispatcher’s understanding of the information, compromise the dispatcher’s judgment during an emergency, increase the error rate for monitoring tasks, and thereby affect safe railway system operation. Therefore, the industry urgently needs a railway dispatcher’s mental workload prediction model that can provide a quantitative basis for optimizing the display interface, thereby maintaining the dispatcher’s mental workload at an appropriate level during dispatch tasks to ensure the safe and efficient operation of the railway dispatch system.

Many scholars have conducted research into mental workload prediction models. For example, Siegel and Wolf developed a human-centered human–computer interactive simulation system that proposed a model to define time pressure as the ratio of the time demand for the performance of a job to the time available, which provides a reference for researching mental workloads from the perspective of time resources. Their time pressure model is now recognized as a fundamental tool for mental workload prediction.[2] Bi and Salvendy used task analysis to develop an operational model that can predict mental workload in a dynamic control system. This model evaluates an operator’s mental workload by a series of systematic indices obtained in the early period of system design, including task fulfillment rate, task complexity, and task uncertainty,[3] which provides insight into analyzing how the mental workload will be affected by different task characteristics. However, quantitative task information methods proposed by the model do not consider the effects of information display format and information display duration on mental workload. Patel used Ohm’s law to describe the human cognitive process to develop and verify a mental workload model based on an individual’s capacity to process information.[4] The formulation of the model is similar to Ohm’s law in that the rate of information transmission, measured by bit/s, is analogous to electric current; the amount of information processed is analogous to voltage; and the function based on an individual’s conditions is analogous to electric resistance. Prediction model considering the impacts of the task information and the characteristics of the operator on the mental workload is not used to predict the mental workload of monitoring task in complex display system as it used only the number of decision alternatives to quantify task information. Some researchers adopted the perspective of cognitive analysis to study mental workload prediction models. Wu and Liu proposed the Queuing Network-Model Human Processor (QN-MHP) model to predict a driver’s mental workload.[5] This model uses a queuing network to simulate the driver’s subjective mental workload and performance of multiple tasks, based on human information processing, comprehensive consideration the effects of information perception subnet, cognitive subnet, and motor subnet on mental workload. However, this model does not take into consideration of the quantitative visual display information. Jo et al. developed a method to quantitatively predict mental workloads through the cognitive architecture Adaptive Control of Thought – Rational (ACT-R),[6] which uses ACT-R cognitive system resources in a given time interval to predict the mental workload. This method offers valuable information for researching mental workloads from the perspective of cognitive resource utilization, but it does not consider the effects of display systems on the operator’s mental workload. These existing models and methods provide valuable insights into mental workload prediction from different perspectives. However, few of them approach the problem from the perspective of monitoring, which would allow us consider the visual behaviors through which a dispatcher obtains information and its impact on mental workload. The feasibility of quantifying information based on the above models is limited; in general, they are not applicable to mental workload prediction for a railway dispatcher because of the wide variety and large amount of information involved in railway dispatch and monitoring tasks.

This study introduces the concept of information entropy into researching the dispatch interface and uses the VACP model [7] to describe the mental workload resulting from obtaining visual information from the interface. This study then proposes a railway dispatcher’s mental workload prediction model that comprehensively considers the dispatcher’s visual behaviors and information exposure; this model can be used for complex visual display systems. To verify the model’s validity, 20 subjects performed dispatch monitoring simulations under different experimental conditions, and three different methods (task performance measure, subjective measure, and physiological measure) were used to evaluate their mental workloads. Finally, the mental workload prediction model was validated by analyzing the correlations between the theoretical prediction values and experimental results.

2. Mental workload prediction model

When a rail emergency occurs, the dispatcher’s main task is to monitor a wide variety of information displayed on the user interface, which continually updates. The dispatcher must process the information promptly as it appears on the display. Based on this system, we can infer that the dispatcher’s mental workload derives mainly from monitoring the visual information on the display. Mental workload can be defined as the ratio of the mental resources required for the performance of a job to the available resources.[8] This study assumes that a dispatcher’s available mental resources per unit time are limited and that monitoring a unit amount of information consumes a certain amount of mental resources, such that the amount of information monitored per unit time can be used to estimate a dispatcher’s mental workload. Consequently, mental workload can be expressed as the ratio of the amount of information H monitored by a dispatcher against the information displaying duration T, the unit is in bit/s: (1)

Shannon used information entropy to conceptualize quantitative information in the field of communication.[9] Hick used the information entropy to measure the complexity of simple decision tasks and proved that subjects’ response times have a linear relation with the complexity of the simple decision task.[10] Liu introduced information theory into measuring traffic sign information; through the experiment, they found that as the amount of traffic sign information increases, the driver's visual fixation time and mental workload increases, and traffic sign comprehension decreases.[11] Liu's measurement method is suitable for the static display information measurement, but is not suitable for dynamic information measurement. These studies show that the information entropy can solve the problems in the field of human-computer interaction. This study uses information entropy to measure the amount of information monitored per unit time. Figure 1 compares the general communication system model [9] with the railway dispatch information transmission model and the classic S-O-R (stimulus–organism–response) cognitive model [12].

Figure 1. Comparison of communication system model, dispatch information transmission model, and S-O-R cognitive model.

As Figure 1 shows, Shannon’s communication system model can be used to describe the process of railway dispatch information transmission. Information sent from information sources refers to the whole railway system, including trains, platforms, and signaling systems. The transmitter is responsible for visually encoding information using a variety of graphics, symbols, and words; for instance, a train is represented by a red rectangle, and traffic information is conveyed through words. After being transformed into a visual format, the information is shown to the dispatcher through the channel, which refers to the dispatch–display interface. The dispatcher obtains visual information from the interface and then applies mental resources to interpret the information and understand the concrete operating situation, and then, the information is passed to the receiver. Finally, the information passes to the destination and the dispatcher takes actions based on his or her understanding. Shannon defined the information as the occurrence of some events that reduce uncertainty. When an emergency occurs, the dispatcher is uncertain of the railway system before the information is perceived in the display interface. When the dispatcher consumes certain mental resources to perceive and comprehend the information in the display interface, the uncertainty of information changes into certainty. Thus, the amount of information in the display interface can be measured by the amount of reduced uncertainty. Information entropy can be used to measure event uncertainty. From the above analysis, we can conclude that we can use information entropy to measure the amount of information on the display interface. Display interface contains a variety of dynamic information, and this study regards each dynamic information unit in the display interface as an information source, then the information entropy formula (2) [9] can be used to calculate the information amount of the information source, where n is the number of information source states, and P_i is the probability of an information source under different states. (2)

The amount of information of the whole display interface can be obtained by adding the amount of information of the single information source based on the additivity principle of information entropy.[13] Figure 1 also shows that the information transmission model only reflects mental resources consumed in the information processing stage, but fails to account for those consumed through the dispatcher’s visual work of obtaining information in the stimulus input stage. So, the information entropy calculation formula can only calculate the mental resources consumed in information processing stages. Different visual behaviors that obtain information require different mental resources that exert different influences on the dispatcher’s mental workload. For instance, a dispatcher obtains information about the distance between different trains by visually judging the distance, whereas dispatchers obtain textual information from warning windows by reading text. These two visual behaviors of obtaining information have different consequences for mental workloads. To take the mental workload consumed in the information input stage into account, we introduced weighted entropy into the calculation of the dispatch interface’s information amount. Guiasu defines the non-negative real number ω as the information weight to indicate the information’s importance.[14] This paper defines ω as the weight of the mental workload level brought about by different visual behaviors of obtaining information in the information input stage. Formula (2) can be transformed into the following: (3)

Weighted entropy comprehensively represents the mental workload caused by the information input stage and the information processing stage. The weighted value ω of the weighted entropy can be represented by the VACP model. The VACP model divides required human information processing resources into four types: visual, auditory, cognitive, and psychomotor; different information processing behaviors consume different resources. The VACP model proposes 28 kinds of basic information processing behaviors and assigns a scale value to each; larger scale numbers indicate behaviors that require more resources to properly process the information.[15] This study invited five railway dispatch experts to categorize visual behaviors of obtaining railway dispatch information by basic visual processing behaviors from the VACP model. Then, each visual behavior of obtaining railway dispatch information was given a corresponding scale value from the VACP model; this enabled us to use the scale value as the weighted value ω of the weighted entropy to distinguish the number of mental resources consumed by different visual behaviors in the information input stage. Table 1 [15] displays the basic visual behavior and the weighted value ω of the weighted entropy:

Table 1. Mental workload for visual behaviors.

Type	Code	Visualbehaviors	Weightedvalue ω	Value of ω afternormalization
Visualbehaviors	1	Visually register/detect	1.0	0.031
	2	Visually discriminate	3.7	0.116
	3	Visually inspect/check	4.0	0.125
	4	Visually locate/align	5.0	0.156
	5	Visually track/follow	5.4	0.169
	6	Visually read	5.9	0.184
	7	Visually scan/search/monitor	7.0	0.219

By substituting H from Formula (3) into (1), we arrive at the following formula: (4)

Before the interface information is obtained by the dispatcher, it is impossible to know the occurring probability of each state of the information source, so we cannot determine the amount of information of each information source. The principle of maximum entropy (POME) provides a statistical inference of unknown probability distribution on the basis of partial knowledge without considering any unknown information.[16] According to POME, we choose the probability distribution of the maximum uncertainty of the information source as the occurring probability of each state of the information source, and then, the amount of information of the information source has the largest objectivity. When the occurring probability of each state of the information source is identical, the information source has the greatest uncertainty. So, we can assume that if the information source has n state types, then the occurring probability of each state of the information source is P = 1/n. Based on this, formula (4) can be transformed into the following: (5)

The dispatcher must monitor various information streams with different information amounts and display durations. For example, in a dispatch task with a large passenger flow, the dispatcher must monitor the distance between trains and warning information, which differ in the amount and display duration. Therefore, this study uses the additivity of information entropy to calculate the mental workload imposed by various information sources in the whole display interface.[13] Assuming that there are a types of information in the display interface, and the amount of each type of information is m, then formula (5) can be transformed into the following: (6)

3. Experimental methods

3.1. Subjects

We recruited 20 postgraduates (13 males and 7 females) from Beijing Jiaotong University to participate in this study. Each had backgrounds in railway traffic knowledge, were right handed, and were between 21 and 25 years old (average 23.6 years, standard deviation 1.19). They had knowledge of the experimental content before beginning the experiment. Their vision or corrected vision was above 1.0, they were not color-blind, and they had no color weaknesses.

3.2. Materials

The experimental interface was designed with reference to the display interface of Beijing’s subway line 1, appropriately simplified and abstracted to meet research needs, as shown in Figure 2. The virtual simulation procedure for the experiment was generated using C# programming language, with operation accuracy and subjective ratings recorded by a computer. The experiment interface was presented on a 23-inch screen with a 1024 × 768 resolution and a 600 lx average experimental illumination. A RED-type desktop eye tracker designed by the German company SMI was used to record subjects’ eye movement, and five-point calibration was conducted before beginning each experiment. During the experiment, the subject looked into the simulation dispatch interface on the screen of the Stimulus PC in front of them. Beneath the screen, a camera took pictures of the subject’s eyes and sent the images to the iView PC testing computer. The iView PC testing computer collected eye movement data at a frequency of 250 Hz.

Figure 2. Experimental interface.

3.3. Experimental design

The experiment simulated a dispatcher’s task of monitoring information on the dispatch interface. Different experimental tasks reflected the need for a dispatcher to monitor and distinguish information about warnings, rail switches, signal lights, and the distance between trains. When errors appeared on the interface, the subject needed to click the corresponding keyboard within a certain time frame. The experiment was designed with a design focus on the amount of information, including five different dispatch tasks, and the information display duration, which was set according to the update time of the interface information. The subject had to finish checking the information before its display ended. The duration of the two levels of displayed information was five seconds. Table 2 shows task information under ten different experimental scenarios. To eliminate the effects of exercise and fatigue, the experimental sequence used a Latin square design.

Table 2. Task information under different experimental scenarios.

Task number	Information content	Number of states	Weighted value	Display duration (s)	Amount of information (bit)
T1H1	Textual information	3	0.184	8	1 × 0.184× log2^3
T1H2	Railway crossings	2	0.031	10	16 × 0.031×log2^2
T1H3	Signal lights	2	0.031	10	33 × 0.031×log2^2
T1H4	Textual information, signal lights	3 2	0.184, 0.031	5, 10	1 × 0.184×log2^3, 33 × 0.031×log2^2
T1H5	Railway crossings, distance between trains	2 3	0.031, 0.116	10, 12	16 × 0.031×log2^2, 10 × 0.116×log2^3
T2H1	Textual information	3	0.184	3	1 × 0.184×log2^3
T2H2	Railway crossings	2	0.031	5	16 × 0.031×log2^2
T2H3	Signal lights	2	0.031	5	33 × 0.031×log2^2
T2H4	Textual information, signal lights	3 2	0.184, 0.031	3, 5	1 × 0.184×log2^3 33 × 0.031×log2^2
T2H5	Railway crossings, distance between trains	2 3	0.031, 0.116	5, 7	16 × 0.031×log2^2 10 × 0.116×log2^3

3.4. Experimental procedure

Before the experiment began, each subject was required to practice to become familiar with the experimental requirements and procedures. During the experiment, the subject was required to closely monitor the interface. Each task condition contained 50 instances of information changes. The subject had to click the keyboard accordingly to respond to the display interface information as soon as possible. After the initial experiment, the system generated a NASA-TLX subjective evaluation table. After filling out the evaluation, the subject took a ten-minute break before the next experiment. Each subject had to complete ten experiments. The eye tracker continuously gathered data during the experiments, and the computer recorded the subject’s responses to information changes.

4. Experimental results and analysis

Theoretical mental workload predictions were calculated for each experiment according to formula (6). Table 3 displays the mental workload prediction values and statistics for all experimental indices under the ten tasks. Analysis of variance (ANOVA) by SPSS indicates that at a significance level of 0.05, the information amount had a significant main effect on accuracy rate (p < .001), NASA-TLX subjective score (p < .001), eye blinking frequency (p < .001), and pupil diameter (p = .001); similarly, the information display duration had a significant effect on accuracy rate (p < .001), NASA-TLX subjective score (p < .001), eye blinking frequency (p = .001), and pupil diameter (p = .027).

Table 3. Modeling and measuring results under different MW conditions (mean ± SD).

MW Condition	Prediction (bit/s)	Accuracy (%)	Subjective Score (point)	Blink Frequency (times/s)	Pupil Diameter (mm)
1	0.0365	98.7 ± 2.2	8.45 ± 2.87	0.281 ± 0.132	3.31 ± 0.36
2	0.0496	93.2 ± 5.3	12.85 ± 3.63	0.248 ± 0.112	3.49 ± 0.36
3	0.0972	91.9 ± 3.9	12.4 ± 4.76	0.216 ± 0.101	3.43 ± 0.38
4	0.0992	86.9 ± 7.8	18.7 ± 6.41	0.217 ± 0.099	3.63 ± 0.44
5	0.1023	78.5 ± 6.7	22.8 ± 5.36	0.178 ± 0.092	3.67 ± 0.4
6	0.1388	76.3 ± 11.8	26.4 ± 6.25	0.162 ± 0.089	3.63 ± 0.42
7	0.2028	71.2 ± 8.1	31.2 ± 6.61	0.127 ± 0.071	3.6 ± 0.43
8	0.2046	73 ± 8.9	31.15 ± 6.86	0.141 ± 0.082	3.66 ± 0.42
9	0.3018	70.3 ± 8.3	32.4 ± 6.11	0.109 ± 0.058	3.74 ± 0.46
10	0.3619	57.3 ± 9.8	41.9 ± 8.77	0.084 ± 0.049	3.86 ± 0.35

Pearson relation analysis was performed between the mental workload predictions and statistics for all experimental indices. It showed that the theoretical prediction had a highly positive correlation with the NASA-TLX subjective score (r = .810, p < .001), a negative correlation with accuracy rate (r= −.786, p < .001), a negative correlation with eye blinking frequency (r = −.524, p < .001), and a weakly positive correlation with pupil diameter (r = .301, p < .001).

5. Discussion and conclusion

This study applied three approaches with four indices to test the mental workload prediction model. The experimental results described in Section 4 demonstrate that this model’s predictions of mental workload changes correlated strongly to the experimental results using different mental workload measurement methods, thereby verifying the mental workload prediction model.

Analysis indicates that mental workload prediction has a medium negative correlation with accuracy rate (r= −.786, p < .001), indicating that as theoretical predictions increase, the amount of information to be processed per unit of time increases and the mental resources allocated to each information unit decreases, resulting in deteriorating accuracy. This result demonstrates that the model can predict changes in performance accuracy on monitoring missions. Wickens’ multiple resource theory [17] explains this outcome by positing that subjects have limited mental resources, which form the basis of various work activities. When the amount of information to be processed per unit time increases, more information compete for limited mental resources, leading to a decline in work performance and accuracy rates. This study measured each subject’s mental workload using the NASA-TLX subjective evaluation table along six dimensions: cognitive load, physical load, time requirement, performance level, intended effort, and frustration degree. NASA-TLX is widely used to measure and diagnose mental workload due to its proven performance.[18] Our analysis showed a highly positive correlation between mental workload theoretical prediction and the NASA-TLX subjective score (r = 0.810, p < .001), indicating that the prediction value reflects the individual’s subjective sense of mental workload in experimental tasks, and that changes in the information amount and information display duration have a significant influence on subjective feelings. In terms of physiological measurements, our analysis showed a negative correlation between theoretical prediction and eye blinking frequency (r= −.524, p < .001), which is regarded as one of the most trustworthy indicators of an individual’s mental workload related to thinking and visual tasks.[19] This result shows that as the amount of information needed to be processed per unit time increases, subjects reduce their blinking frequency by shortening the time blinking, and prolonging the duration between blinks; this may promote good performance by helping subjects pay more attention and allocate more mental resources to visual information on the interface. Some research suggests that while completing cognitive tasks, increases in mental workload can increase the pupil’s diameter.[20] This study found a weakly positive correlation between mental workload prediction and pupil diameter (r = .301, p = .027). A positive correlation indicates that as the amount of information processed per unit time increases, the information processing load increases; to complete tasks, the subject exerts more effort to monitor information in the interface, resulting in increased pupil diameter. However, the analysis did not reveal a high correlation, possibly due to other factors that can influence changes in pupil diameter. For example, when mental workload reaches a certain level, pupil diameter may decrease because of the subject’s fatigue.[21] Additionally, psychological tension may influence changes in pupil diameter. Consequently, pupil diameter must be used in conjunction with other related measurements to study mental workload.

In conclusion, this study introduced the concept of information entropy and applied the VACP model in the form of weighted entropy to measure the amount of information in a dispatch interface. These concepts contributed to a new prediction model for a dispatcher’s mental workload, which comprehensively considers the visual behaviors required to obtain information. The proposed model can predict the mental workload involved in monitoring tasks on a complicated display interface that contains various forms of information. This study can provide a reference for future research into human factors that contribute to optimizing human–computer interactive interfaces and evaluations.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Funding

This work was supported by Beijing Municipal Science & Technology Commission, followed by the grant number [D161100000116002].