Empirical investigation of workloads of operators in advanced control rooms

Abstract

This paper compares the workloads of operators in a computer-based control room of an advanced power reactor (APR 1400) nuclear power plant to investigate the effects from the changes in the interfaces in the control room. The cognitive–communicative–operative activity framework was employed to evaluate the workloads of the operator's roles during emergency operations. The related data were obtained by analyzing the tasks written in the procedures and observing the speech and behaviors of the reserved operators in a full-scope dynamic simulator for an APR 1400. The data were analyzed using an F-test and a Duncan test. It was found that the workloads of the shift supervisors (SSs) were larger than other operators and the operative activities of the SSs increased owing to the computer-based procedure. From these findings, methods to reduce the workloads of the SSs that arise from the computer-based procedure are discussed.

Keywords:

• workload
• computer-based procedures
• computer-based control room
• error
• experiment
• performance

1. Introduction

With the technical improvements of computer hardware and software, computer-based control rooms of nuclear power plants (NPPs) have been actively studied [1–3]. Exemplary features of computer-based advanced control rooms are computer-based procedures (CBPs), soft controls, and large display panels [4]. Operators in advanced control rooms use computer interfaces of these features to operate the plants.

Owing to changes of the interfaces, interaction patterns between operators and machines have also changed. For example, while board operators (BOs) in conventional control rooms walk up to the valve button to open a valve, BOs in advanced control rooms select an item on a screen using a digital device while sitting in their chairs. As Kim et al. reported, the cognitive behaviors and communication patterns of operators have also changed [5]. Because the shift supervisors (SSs) in advanced control rooms can access the plant information, it was found that the SS-initiated inquiry patterns increased.

From the changes in the physical and mental behaviors of the operators, we can expect that the workloads of operators in advanced control rooms can also be changed. BOs additionally perform secondary tasks that navigate information in fixed screens by converting many windows. SSs should also use soft controls in CBPs. Therefore, in this paper, we empirically investigated how the workloads of operators in advanced control rooms are changed. We were particularly interested in whether there are unbalanced workloads between operators or excessive workloads of specific operator roles. It is expected that this investigation will provide important insight into the design of interactions in a digital environment and the roles of operators.

To investigate the workloads of operators, we used a cognitive–communicative–operative activity (COCOA) framework [6]. A computer-based full-scope dynamic simulator for an APR 1400 (advanced power reactor), which is under construction in Korea, was employed as a computer-based advanced control room. In this control room, we analyzed the workloads of the operators during emergency situations and compared the results between the roles of the operators.

This paper is organized as follows. Section 2 briefly reviews the methodologies to evaluate the work-loads and introduces the COCOA methods employed in this study. Section 3 describes an experiment using the COCOA framework. Section 4 describes the results of the experiment. Section 5 interprets results of the experiment and discusses insights on the unequal distribution of workloads between operators. Finally, we summarize our conclusions of this study in Section 6.

2. Related work

2.1. Overview

Workload measurement techniques can be typically organized into four broad categories: subjective workload measures, performance-based measures, physiological measures, and task loading methods [7,8]. The subjective workload measures evaluate the workloads on the basis of subjective experience for the demanded tasks. It primarily uses self-reporting questionnaires which ask the beliefs, values, preferences, and attitudes of the respondents. The performance-based measures analyze the workloads from the speed, accuracy, or error rate of the demanded tasks. In some cases, one or two additional tasks are required to the subjects for deeply understanding the availability of the cognitive resources. The physiological measures record the physiological signals that are strongly associated with the subjective experience of the operators. The task loading methods explicitly calculate the demands of the given tasks that can exert environmental or psychological influence on the operators.

Among the methods used to evaluate the workload, the COCOA method, which is a kind of task loading method, was selected for this study. The reasons for this are as follows. Our experiment was conducted to compare the workloads for all operators, who have different tasks under the same emergency situations. Because various emergency tasks are intermittently performed, the performance-based measures, which assess the operator's performance for a small number of tasks, are not easy to be applied to this study. The physiological measures, which use measurement equipment, can be intrusive to the emergency tasks, because operators in the control room dynamically behave for operating devices or communicate with other operators. Furthermore, the subjective questionnaire measures as well as the performance-based and physiological approaches require a large amount of data to evaluate the system characteristics related to the workloads, because data of these approaches basically depend on subjective experience such as ability, familiarity, attitude, or emotional state of the participants. Because many features of an advanced control room have been newly developed, and some interfaces or usage of the systems have been continuously improved, the task-loading method, which prospectively estimates workloads with a small amount of data are more adequate to this study.

2.2. COCOA framework

The COCOA framework was recently developed to analytically evaluate the workloads of operators in computer-based control rooms [6]. This framework provides the taxonomy of three dimensions of operator activities: cognition, communication, and operative activities (Table 1). The cognitive activities are based on definitions of cognitive activities in CORA (Cognitive Reliability Analysis) method, which has been developed to assess the cognitive demands of NPP operators [9]. From the list of cognitive activities in CORA, an activity, communication, was removed and extended into the communicative dimension of the COCOA framework. The communicative activities are based on definitions of an extended speech act coding scheme that has been developed to analyze the communication patterns of NPP operators [10]. From the list of definitions in the scheme, four activities, which are typically performed during emergency situations, are counted in the COCOA framework. The operative activities are defined by operations using digital devices of the main control rooms in an APR 1400, such as a mouse, touch screen, and buttons.

Table 1. Operator's activities of the COCOA framework.

Dimension	Activity	Description
Cognitive activity	COMPARE	Comparing two or more entities of system states.
	DIAGNOSE	Recognizing or determining the cause of the system states by signals or parameters.
	EVALUATE	Checking a system state with consideration of other system parameters or states.
	EXECUTE	Performing a single prescribed action (e.g. open, close, turn on, etc.).
	MAINTAIN	Sustaining a specific system state by executing or regulating systems.
	MONITOR	Continuously observing the system states or parameter.
	RECORD	Writing down or logging the system states or events.
	REGULATE	Changing the quantity, speed, or direction of the system parameters or states.
	VERIFY	Checking an entity of system states.
	SCAN	Briefly reviewing specific system states by displays or other information.
	PLAN	Formulating a path to achieve specific goals.
	IDENTIFY	Recognizing the overall state of a specific system.
Communicative activity	COMMAND	Ordering an operator to execute or regulate a system or component.
	COMMAND-ACK	Informing whether a listener understood a command.
	INQUIRY	Asking about system states or parameters.
	REPLY	Answering a question.
Operative activity	SWITCH---SCR	Altering a display screen to read or operate specific parameters or states.
	OPEN---CTRLPNL	Opening a control panel to operate a specific system.
	CLOSE---CTRLPNL	Closing a control panel that has been opened.
	CLICK---EXECUTE	Pushing a button to operate a single entity (e.g. opening a valve).
	CLICK---REGU---UPDN	Pushing a direction button to regulate a system parameter or state (ex. opening a valve).
	CLICK---REGU---FASTUPDN	Pushing a double-direction button to regulate a system parameter or state (ex. opening a valve).
	CLICK---ENABLED	Pushing a button to enable a control panel to be usable.
	CLICK---MANNUAL	Pushing a button to manually operate a single entity.
	CONF---STEP	Clicking a button confirming that all instructions of a step have been completed.
	CONF---SUBSTEP	Clicking a button confirming that an instruction of a step has been completed.
	ACK---ALARM	Checking and silencing a notifying alarm.
	CONF---CHANN	Pushing a button to verify the channel of an entity to be operated.

To calculate the demands of a task, the numbers of activities that have been performed during a given situation are counted. Procedures are basically used to analyze the activities of the situation. Two or more video-audio records for the situation are also sometimes used to identify who typically conducts a specific activity. The numbers of operative activities are calculated by tracking the operator behaviors from video records or using a system that automatically collects the operation logs.

3. Analysis using COCOA framework

We compared the workloads of operators in the main control room of an APR 1400 by the COCOA framework. The operation team of an APR 1400 has five roles: an SS, reactor operator (RO), turbine operator (TO), electric operator (EO), and shift technical assistant (STA). Each role has a similar responsibility and authority to that of a conventional control room. That is, an SS generally progresses a procedure by obtaining information from BOs or instruments, and gives directions to other operators during an emergency situation. BOs such as RO, TO, and EO usually recognize and report the plant situations and take actions that are directed by the SS. Table 2 shows an example of typical discourses and behaviors of SS and BO. An STA independently monitors the critical functions of the plant.

Table 2. An example of discourses and behaviors of operators in APR 1400.

Procedure instruction	SS	BO (RO)
LOCA step # 4.1) Is the pressure of the pressurizer less than 127.3 kg/cm^2A?	‘RO, is the pressure of the pressurizer lower than 127.3 kg/cm^2A?’	‘Yes’
		<Checks the pressure> ‘Yes, the pressure is lower than 127.3 kg/cm^2.’
	‘Yes, I confirmed’ <Clicks an instruction confirmation button of CBP>
LOCA step # 4.2) Confirm that the safety injection actuation signal (SIAS) was initiated.	‘RO, was the SIAS initiated?’	‘Yes’
		<Checks the SIAS> ‘Yes, the SIAS was initiated.’
	‘Yes, I confirmed’ <Clicks an instruction confirmation button of CBP>

Each operator in a computer-based control room uses workstation-based information systems, which deliver information by four monitors. Two monitors mainly show states of components or systems of the plant. And other two monitors are supposed to display alarm lists and CBPs. All operators can control the plants by soft controllers. When an operator adjusts a plant parameter, the operator finds a window showing the parameter by clicking a digital interface such as a mouse. In addition, after the operator selects the relevant figure of the component, a control panel to control the parameter is opened. The operator can change the parameter by a mouse or touch screen.

The CBP, which is provided to all operators, is basically managed by an SS during emergency situations. The SS reads instructions on the CBP and takes an action as instructed. After the SS completed an instruction, the SS records that the instruction has been checked by highlighting the circle in the instruction box (refer to Table 2). In addition, after all instructions in a step are checked, the SS reviews all instructions and clicks a completion button to move the next step. An RO, TO, EO, and STA can watch how the SS progresses the procedure by the CBP screen of their own monitor. The contents of the CBP were written by the emergency operating guidelines, CEN-152, Revision 5.2, of CE-type NPPs [11], and are practically same as the procedures of an OPR 1000, which is a reference plant of an APR 1400.

In addition, the computer-based control room has a large display panel that presents the overall states and parameters of the plant systems, and an advanced alarm system that presents information of activated alarms.

The operators in the three operation teams, A, B, and C have been trained to operate an APR 1400 through the full scope simulator, and many of them have work experience in conventional power plants; hence, they are familiar with the contents of the procedures and systems of the plants. The four members for each operation team, SS, RO, TO, and EO participated in this experiment.

The workloads of the operators in the APR 1400 were evaluated by the COCOA framework for two emergency situations; a LOCA (loss of coolant accident) and an SGTR (steam generator tube rupture). To analyze the operator behaviors under the LOCA situation, a situation of that a cold leg in a primary coolant loop was totally broken was simulated. For the SGTR scenario, a tube of the first steam generator was ruptured at the beginning of the simulation.

Cognitive and communicative activities for each scenario were determined in accordance with the emergency operating procedures for these scenarios. To obtain who and how much each operator conducted cognitive, communicative, and operative activities, we experimented three operator teams for the LOCA and SGTR scenarios in a full scope simulator of the main control room of an APR 1400. The operators performed all tasks of the standard post-trip action (SPTA) procedure and diagnostic action (DA) procedure, and early tasks of optimal recovery procedure such as LOCA and SGTR procedures for the given scenarios (LOCA, step numbers 1–18; SGTR, step numbers 1–16). All operators’ behaviors, speeches, and related screens of all operators were recorded by four video cameras and a microphone. From these records, how many activities of the COCOA framework were conducted by each operator was calculated. If there was a difference in cognitive, communicative, or operative activities between each operator, the average frequency of the activities was used.

Analysis of variance (ANOVA) and Duncan's multiple range tests were employed to statistically analyze the differences between the frequencies of operator roles. The ANOVA test appraises whether or not the averages of two or more groups are equal. If the averages are not equal, the Duncan test identifies which groups are not equal by comparing all pairs of groups.

4. Results

The frequency by the activity types and operator roles is shown in Figures 1 and 2. Figure 1 shows the results of LOCA scenarios and Figure 2 shows the results of the SGTR scenarios. The tables in the figures indicate the frequency of activity during following the procedures of SPTA, DA, and optimal recovery procedure (LOCA or SGTR procedure). In addition, the bar graphs show the total frequency of activity. The different letters over the bars indicate significant differences of frequencies between different roles of operators for each activity according to ANOVA and Duncan tests (p < 0.05) [12]. For example, the frequency of cognitive activities of SS is grouped as ‘a’, and the frequency of activities of RO is grouped as ‘b’, in Figure 1. This implies that the frequencies of two operator roles are significantly different. However, all frequencies of operative activities in Figure 1 are grouped as ‘a’. This means that the differences between operative activities of all operator roles are not statistically significant. Table 3 shows the F ratios resulted from the ANOVA of LOCA and SGTR scenarios.

Table 3. F ratios of ANOVA conducted to compare activity frequencies between operator roles.

	F ratio
Dimension	LOCA	SGTR
Cognitive	414.5^***	368.9^***
Communicative	87.58^***	401.2^***
Operative	3.463	6.009^*

^***p < 0.001; ^*p < 0.05.

Figure 1. COCOA activity frequency during LOCA scenario.

Figure 2. COCOA activity frequency during SGTR scenario.

According to Figure 1, the SSs most frequently conducted the cognitive and communicative activities during the LOCA scenario. The ROs were second, followed by the EOs and TOs. The differences between the SSs and other operators were remarkable in this figure. The operative activities of the SSs were also the most often found. The EOs were second, followed by the TOs and ROs. Meanwhile, Figure 2 shows that the SSs most frequently conducted the cognitive and communicative activities during the SGTR scenario. The frequency of the SS activities was outstanding than other operators. The ROs were second, followed by the TOs and EOs. The operative activities of the SSs were also the most often found. It was followed by the TOs, ROs, and EOs.

Figures 1 and 2 commonly show that the SSs conducted more cognitive, communicative, and operative activities than other operators. The ROs conducted the next most frequent activities of cognitive and communication. However, the TOs conducted more operative activities than the ROs. The noticeable differences between Figures 1 and 2 are the frequencies of TO activities. In the SGTR scenario (Figure 2), it was also shown that the TOs’ activities were relatively increased, especially, the operative activities. This was because the SGTR procedure requires more control tasks related to the secondary system than the LOCA scenario. The EOs’ operative activities were frequently observed in the LOCA scenario.

The results by the Duncan test also reveal that the SSs significantly conducted more cognitive and communicative activities. In the SGTR scenario, the SSs’ operative activities were more than TO and EO. However, the frequencies of the operative activities between the roles of the operators were not significantly different in the LOCA scenario. Although the mean differences in the operative activities between the roles of the operators were more than in other activities, the Duncan multiple range tests did not reject the idea that the mean differences between the roles of the operators are significant because the standard deviations of operative activities were large (refer to Table 4). It was because the operators had not only different strategies to manipulate the systems, but diverse usages to handle CBP or soft controllers.

Table 4. Standard deviations of activities for each activity type and operator role.

		SS	RO	TO	EO
LOCA	Cognitive	2.9	6.8	1.2	2.6
	Communicative	8.4	5.7	2.4	0.8
	Operative	9.9	16.0	12.3	47.5
SGTR	Cognitive	6.6	2.6	1.4	0.5
	Communicative	1.7	0.8	3.6	0.8
	Operative	9.9	39.0	14.7	24.2

Among the cognitive activities, the SSs primarily used ‘VERIFY’ (66%) and ‘SCAN’ (30%) activities. The ROs primarily used ‘VERIFY’ (40%) and ‘EVALUATE’ (32%) activities. The TOs primarily used ‘VERIFY’ (28%), ‘EXECUTE’ (22%), and ‘EVALUATE’ (21%). Finally, the EOs primarily used ‘EXECUTE’ (34%), ‘VERIFY’ (30%), and ‘EVALUATE’ (23%).

Among the communicative activities, while the SSs primarily used an ‘INQUIRY’ (78%) activity, the other operators primarily used a ‘REPLY’ (RO-89%, TO-70%, EO-84%).

Among the operative activities, the SSs primarily used ‘CONF_---SUBSTEP’ (63%) and ‘CONF_---STEP’ (25%) activities in both scenarios. The ROs primarily used ‘SWITCH_---SCR’ (56%) and ‘CLICK_---REGU_---FASTUPDN’ (22%). The TOs primarily used ‘CLICK_---EXECUTE’ (25%), ‘SWITCH_---SCR’ (24%), ‘OPEN_---CTRLPNL’(16%), and ‘CLICK_---REGU_---FASTUPDN’ (14%). The EOs primarily used ‘SWITCH_---SCR’ (41%), ‘OPEN_---CTRLPNL’(20%), and ‘CLOSE_---CTRLPNL’ (18%). The ratios of the primary activities, such as ‘CLICK_---EXECUTE’ or ‘CLICK_---REGU_---UPDN’ that directly affect the system states among all operative activities of each role, were 0% at SS, 30% at RO, 44% at TO, and 15% at EO.

5. Discussion

5.1. Activities and workloads of operators

In this paper, we compared the workloads between operator roles in an APR 1400 using the COCOA framework. Basically, all operators performed the defined cognitive functions of their roles. The SSs usually asked about some parameters or states of the plant to the BOs, and the BOs verified or compared the information and replied to them. For this reason, the SSs showed many ‘INQUIRY’ activities and BOs conducted lots of ‘REPLY’ activities. Although all operators primarily used ‘VERIFY’ activities, BOs conducted ‘VERIFY’ activities to check the system status from the instruments, while SSs mainly conducted ‘VERIFY’ activities to confirm the BO reports. When the SSs instructed to execute or regulate some system functions, the BOs followed the instructions. Hence, the BOs showed many ‘EXECUTE’ activities to control the plants. The SSs sometimes obtained or verified information from the workstation based information systems or large display panel without communicating with the BOs.

The bar graphs and Duncan test of frequency by the activity types and operator roles revealed that the SSs had a larger amount of workload than the BOs. In particular, the SSs showed a large amount of workload for following the procedures. Most cognitive activities and operative activities of the SSs were observed during the use of the CBP. For example, most ‘VERIFY’ and ‘SCAN’ activities among the cognitive activities of the SSs were performed to check and confirm the directions during the procedures. The activities of clicking buttons such as ‘CONF_---SUBSTEP’ and ‘CONF_---STEP’ to confirm the directions made up 88% of the operative activities of the SSs. The cognitive activities can be similarly observed in conventional control rooms, because the SSs in conventional control rooms are supposed to check all directions of the procedure during emergency operations. However, the operative activities of the SSs in advanced control rooms are strongly related to behaviors that use the soft controller to handle the CBP.

It is noteworthy that the average frequencies of operative activities of SSs were larger than other operators. The Duncan test to compare the frequency in SGTR scenario also showed that there are significant differences between the SSs and other operators. This implies that handling the CBP requires a large amount of activities of the SSs and makes difference or workloads between advanced and conventional control rooms.

The BOs’ operative activities were also frequently observed, and over 50% of them were secondary operations such as ‘SWITCH_---SCR’, which do not directly affect the system states. However, because all BOs in conventional control rooms generally move to a controller to operate the controller, while the BOs in the advanced control room sit in front of the workstations and click some items on the displays, it is not clear how much the secondary operations additionally affected the BOs’ subjective workloads compared with moving activities in conventional control rooms.

5.2. Insights

Some safety functions or techniques that are newly introduced to prevent human errors may produce additional workloads of the operators. This effect also can reduce the performance of the operators or impede the safety of the system [13]. Therefore, when a safety function is applied, a trade-off between reliability improvement from the function and additional workload should be considered.

The CBP in an APR 1400 prevents operators from arbitrarily following emergency operating procedures and aids in carefully managing the procedures by forcing operators to confirm and click steps and substeps of the procedures [14,15]. In this study, however, it was shown that the CBP affects the operator's workloads. The SSs in the advanced control room should follow the procedures as well as monitor and manage the system status and behaviors of other operators. In this situation, the CBP required more operative activities from SSs than other operators. Hence, to sufficiently secure the cognitive and physical resources of the SSs, it is necessary to reduce the workloads of the SSs that come from the CBP. For example, an intelligent system that automatically evaluates and checks the system states or parameters written in the procedures can be developed [16]. If an intelligent system is attempted to be developed, the reliability of the system should be sufficiently ensured and the operators should be educated to certainly monitor the system. It is also possible to make one or more BOs handle the CBP during some steps or substeps in the procedures. To do so, it is important to educate the BOs to sufficiently understand the system and correctly follow the procedures. In addition, who should handle the CBP and when, and how to change the handler from one operator to another, should be clearly defined.

5.3. Limitations

The statistical power of the ANOVA and Duncan tests can be improved by more experiment data. The COCOA framework was intrinsically developed to prospectively evaluate workloads using small number of experiments or definite task analysis. The results of the tests showed how reliably the activities can be compared by the framework with small amount of data. However, to obviously understand the differences of workloads, particularly about operative activities, more experiments can be suggested.

The Duncan test of frequency by the activity types and operator's roles showed significant differences between the operator's roles in cognitive and communicative activities. This was because the operator's tasks are clearly defined for each role. However, the difference between the operator's roles in the operative activities of the LOCA scenario was not significant, although the mean differences between them were larger than the other types of activities. As mentioned previously, the operators use soft controllers diversely and have different manipulation methods. It is necessary to establish guidance and education for the efficient operation of the interfaces.

The frequencies of each activity of the COCOA framework were employed in this study. There are many different activities, and each activity can differently affect the operator's subjective workloads. For example, regulating the pressure of a reactor coolant system does not have the same influence as recording the accident time. However, these differences were not considered during the calculation of the workloads. As further work, the weighting factors of the activity for estimating the subjective workloads can be evaluated by expert judgments and pairwise comparisons [17].

6. Conclusion

In this study, we empirically evaluated the workloads of operators in advanced control rooms and compared the workloads between operator roles using the COCOA evaluation method. The behaviors of the operators in three operation teams were observed during both LOCA and SGTR scenarios, and the frequencies of the activities, defined in the COCOA framework, were compared. As a result, it was found that the SSs conducted more activities than other operator roles, and the CBP requires the SSs to conduct lots of operative activities. We discussed some alternative strategies to support the SSs to easily afford an understanding and management of the overall plant situation. We plan to apply a strategy to reduce the SS workloads and compare these workloads with the results of this study.

Additional information

Funding

This work was supported by Nuclear Research & Development Program of the National Research Foundation of Korea grant, funded by the Korean government, Ministry of Science, ICT & Future Planning [grant number 2013M2A8A4025991].