Development and validation of a task load index for process control room operators (PCRO-TLX)

References

• Ayre, C., and A.J. Scally. 2014. “Critical Values for Lawshe’s Content Validity Ratio: Revisiting the Original Methods of Calculation.” Measurement and Evaluation in Counseling and Development 47 (1): 79–86. doi:10.1177/0748175613513808.
   Web of Science ®Google Scholar
• Ayres, P., J.Y. Lee, F. Paas, and J.J. van Merriënboer. 2021. “The Validity of Physiological Measures to Identify Differences in Intrinsic Cognitive Load.” Frontiers in Psychology 12: 702538. doi:10.3389/fpsyg.2021.702538.
   PubMed Web of Science ®Google Scholar
• Braarud, P.O. 2020. “An Efficient Screening Technique for Acceptable Mental Workload Based on the NASA Task Load Index-Development and Application to Control Room Validation.” International Journal of Industrial Ergonomics 76: 102904. doi:10.1016/jergon.2019.102904.
   Web of Science ®Google Scholar
• Bruya, B., and Y.-Y. Tang. 2018. “Is Attention Really Effort? Revisiting Daniel Kahneman’s Influential 1973 Book Attention and Effort.” Frontiers in Psychology 9: 1133. doi:10.3389/fpsyg.2018.01133.
   PubMed Web of Science ®Google Scholar
• Cain, B. 2007. A Review of the Mental Workload Literature. Technical report. Toronto: Defence Research and Development.
   Google Scholar
• Charles, R.L., and J. Nixon. 2019. “Measuring Mental Workload Using Physiological Measures: A Systematic Review.” Applied Ergonomics 74: 221–232. doi:10.1016/j.apergo.2018.08.028.
   PubMed Web of Science ®Google Scholar
• Chuang, C.-Y., C.J. Lin, W.-J. Shiang, T.-L. Hsieh, and J.-L. Liou. 2016. “Development of an Objective Mental Workload Assessment Tool Based on Rasmussen’s Skill–Rule–Knowledge Framework.” Journal of Nuclear Science and Technology 53 (1): 123–128. doi:10.1080/00223131.2015.1027156.
   Web of Science ®Google Scholar
• Cousins, R., and C. Donnell. 2012. “Nurse Prescribing in General Practice: A Qualitative Study of Job Satisfaction and Work-Related Stress.” Family Practice 29 (2): 223–227. doi:10.1093/fampra/cmr077.
   PubMed Web of Science ®Google Scholar
• Das, L., M.U. Iqbal, P. Bhavsar, B. Srinivasan, and R. Srinivasan. 2018. “Toward Preventing Accidents in Process Industries by Inferring the Cognitive State of Control Room Operators through Eye Tracking.” ACS Sustainable Chemistry & Engineering 6 (2): 2517–2528. doi:10.1021/acssuschemeng.7b03971.
   Web of Science ®Google Scholar
• Dehais, F., A. Lafont, R. Roy, and S. Fairclough. 2020. “A Neuroergonomics Approach to Mental Workload, Engagement and Human Performance.” Frontiers in Neuroscience 14, 268. doi:10.3389/fnins.2020.00268.
   PubMed Web of Science ®Google Scholar
• Dekker, S., J.M. Nyce, R. Van Winsen, and E. Henriqson. 2010. “Epistemological Self-Confidence in Human Factors Research.” Journal of Cognitive Engineering and Decision Making 4 (1): 27–38. doi:10.1518/155534310X495573.
   Google Scholar
• Desaulniers, D.R. 1997. “Stress in the Control Room: Effects and Solutions [Nuclear Power Plants].” Proceedings of the 1997 IEEE Sixth Conference on Human Factors and Power Plants, 1997. ‘Global Perspectives of Human Factors in Power Generation’, Orlando, FL, USA. doi:10.1109/HFPP.1997.624894.
   Google Scholar
• De Waard, D., and B. Lewis-Evans. 2014. “Self-Report Scales Alone Cannot Capture Mental Workload.” Cognition, Technology & Work 16 (3): 303–305. doi:10.1007/s10111-01400277-z.
   Web of Science ®Google Scholar
• De Winter, J.C.F. 2014. “Controversy in Human Factors Constructs and the Explosive Use of the NASA-TLX: A Measurement Perspective.” Cognition, Technology & Work 16 (3): 289–297. doi:10.1007/s10111-014-0275-1.
   Web of Science ®Google Scholar
• Di Stasi, L.L., A. Antoli, M. Gea, and J.J. Cañas. 2011. “A Neuroergonomic Approach to Evaluating Mental Workload in Hypermedia Interactions.” International Journal of Industrial Ergonomics 41 (3): 298–304. doi:10.1016/j.ergon.2011.02.008.
   Web of Science ®Google Scholar
• Ersin, K., O. Gundogdu, S.N. Kaya, D. Aykiri, and M.B. Serbetcioglu. 2021. “Investigation of the Effects of Auditory and Visual Stimuli on Attention.” Heliyon 7 (7): e07567. doi:10.1016/j.heliyon.2021.e07567.
   PubMed Web of Science ®Google Scholar
• Ghanavati, F.K., A. Choobineh, S. Keshavarzi, A.A. Nasihatkon, A.S.J. Roodbandi. 2019. “Assessment of Mental Workload and Its Association with Work Ability in Control Room Operators.” La Medica de Lavaro 110: 389–397. doi:10.23749/mdl.v110i5.8115.
   PubMed Web of Science ®Google Scholar
• Hancock, G. M., L. Longo, M. S Young, and P. A. HancocK. 2021. “Mental Workload”. In Handbook of Human Factors and Ergonomics, edited by G. Salvendu and W. Karwowski, 203–226. John Wiley and Sons, Inc. doi:10.1002/9871119636113.ch7.
   Google Scholar
• Harris, D., M. Wilson, and S. Vine. 2020. “Development and Validation of a Simulation Workload Measure: The Simulation Task Load Index (SIM-TLX).” Virtual Reality 24 (4): 557–566. doi:10.1007/s10055-019-00422-9.
   Web of Science ®Google Scholar
• Hart, S.G. 2006. “NASA-Task Load Index (NASA-TLX); 20 Years Later.” Proceedings of the Human Factors Ergonomics Society Annual Meeting, Vol. 50, 904–908. doi:10.1177/154193120605000909.
   Google Scholar
• Hart, S.G., and L.E. Staveland. 1988. “Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research.” In Human Mental Workload. Advances in Psychology, Vol. 52, edited by P.A. Hancock and N. Meshkati, 139–183. North-Holland. doi:10.1016/S0166-4115(08)62386-9.
   Google Scholar
• Holsti, O.R. 1969. Content Analysis for the Social Sciences and Humanities. Reading, MA: Addison-Wesley.
   Google Scholar
• Kahneman, D. 1973. Attention and Effort. Englewood Cliffs, NJ: Prentice-Hall.
   Google Scholar
• Lawshe, C.H. 1975. “A Quantitative Approach to Content Validity.” Personnel Psychology 28 (4): 563–575. doi:10.1111/j.1744-6570.1975.tb01393.x.
   Web of Science ®Google Scholar
• Li, X., D.J. McKee, T. Horberry, and M.S. Powell. 2011. “The Control Room Operator: The Forgotton Element in Mineral Process Control.” Minerals Engineering 24 (8): 894–902. doi:10.1016/j.mineng.2011.04.001.
   Web of Science ®Google Scholar
• Longo, L. 2015. “A Defeasible Reasoning Framework for Human Mental Workload Representation and Assessment.” Behavioural and Information Technology 34 (8): 758–786. doi:10.1080/0144929X.2015.1015166.
   Web of Science ®Google Scholar
• Lundgrén-Laine, Heljä, and Sanna Salanterä. 2010. “Think-Aloud Technique and Protocol Analysis in Clinical Decision-Making Research.” Qualitative Health Research 20 (4): 565–575. doi:10.1177/1049732309354278.
   PubMed Web of Science ®Google Scholar
• Luximon, A., and R.S. Goonetilleke. 2001. “Simplified Subjective Workload Assessment Technique.” Ergonomics 44 (3): 229–243. doi:10.1080/00140130010000901.
   PubMed Web of Science ®Google Scholar
• Marras, W.S. 2006. Fundamentals and Assessment Tools for Occupational Ergonomics. CRC Press. doi:10.1201/9781420003635.
   Google Scholar
• Matthews, G., J. De Winter, and P.A. Hancock. 2020. “What Do Subjective Workload Scales Really Measure? Operational and Representational Solutions to Divergence of Workload Measures.” Theoretical Issues in Ergonomics Science 21 (4): 369–396. doi:10.1080/1463922X.2018.1547459.
   Web of Science ®Google Scholar
• Matthews, G., L.E. Reinerman-Jones, D.J. Barber, and J. Abich IV. 2015. “The Psychometrics of Mental Workload: Multiple Measure Are Sensitive but Divergent.” Human Factors 57 (1): 125–143. doi:10.1177/0018720814539505.
   PubMed Web of Science ®Google Scholar
• Mitchell, D.K. 2000. Mental Workload and ARL Modelling Tools. ARL-TN-161. Aberdeen Proving Ground, MD: Army Research Laboratory.
   Google Scholar
• Mohammadian, M., H. Parsaei, H. Mokarami, and R. Kazemi. 2022. “Cognitive Demands and Mental Workload: A Filed Study of the Mining Control Room Operators.” Heliyon 8 (2): e08860. doi:10.1016/j.heliyon.2022.e08860.
   PubMed Web of Science ®Google Scholar
• Moray, N. 1979. “Models and Measures of Mental Workload.” In Mental Workload. NATO Conference Series, Vol. 8, edited by N. Moray. Boston, MA: Spriner. doi:10.1007/978-1-4757-0884-4_2.
   Google Scholar
• Ntuen, C.A. 1999. “The Application of Fuzzy Set Theory to Cognitive Workload Evaluation of Electronic Circuit Board Inspectors.” Human Factors and Ergonomics in Manufacturing 9 (3): 291–301. doi:10.1002/9SICI1520-6564(199922)9:3<291::AID-HRM6>3.0.CO;2-S.
   Google Scholar
• Pauzié, A. 2008. “A Method to Assess the Driver Mental Workload: The Driving Activity Load Index (DALI).” IET Intelligent Transport Systems 2 (4): 315–322. doi:10.1049/iet-its:20080023.
   Web of Science ®Google Scholar
• Polit, D.F., C.T. Beck, and S.V. Owen. 2007. “Is the CVI an Acceptable Indicator of Content Validity? Appraisal and Recommendations.” Research in Nursing & Health 30 (4): 459–467. doi:10.1002/nur.20199.
   PubMed Web of Science ®Google Scholar
• Pretorius, A., and P. Cilliers. 2007. “Development of a Mental Workload Index: A Systems Approach.” Ergonomics 50 (9): 1503–1515. doi:10.1080/00140130701379055.
   PubMed Web of Science ®Google Scholar
• Reid, G.B., and T.E. Nygren. 1988. “The Subjective Workload Scaling Technique: A Scaling Procedure for Measuring Mental Workload.” In Human Mental Workload. Advances in Psychology, Vol. 52, edited by P.A. Hancock and N. Meshkati, 185–218. North-Holland. doi:10.1016/S0166-4115(08)62387-0.
   Google Scholar
• Rönkkö, M., and E. Cho. 2022. “An Updated Guideline for Assessing Discriminant Validity.” Organizational Research Methods 25 (1): 6–14. doi:10.1177/1094428120968614.
   Web of Science ®Google Scholar
• Rubio, S., E. Díaz, J. Martín, and J.M. Puente. 2004. “Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA‐TLX, and Workload Profile Methods.” Applied Psychology 53 (1): 61–86. doi:10.1111/j.1464-0597.2004.00161x.
   Web of Science ®Google Scholar
• Schumacher, S., M. Kleinmann, and K.G. Melchers. 2011. “Job Requirements for Control Room Jobs in Nuclear Power Plants.” Safety Science 49 (3): 394–405. doi:10.1016/jssci.2010.10.002.
   Web of Science ®Google Scholar
• Tao, D., H. Tan, H. Wang, X. Zhang, X. Qu, and T. Zhang. 2019. “A Systematic Review of Physiological Measures of Mental Workload.” International Journal of Environmental Research and Public Health 16 (15): 2716. doi:10.3390/ijerph16152716.
   PubMed Web of Science ®Google Scholar
• Van Acker, B. 2020. “Mental Workload Monitoring in the Manufacturing Industry: Conceptualisation, Operationalisation and Implementation.” Doctoral diss., Ghent University.
   Google Scholar
• Van Acker, B.B., D.D. Parmentier, P. Vlerick, and J. Saldien. 2018. “Understanding Mental Workload: From a Clarifying Concept Analysis toward an Implementable Framework.” Cognition, Technology & Work 20 (3): 351–365. doi:10.1007/s10111-018-0481-3.
   Web of Science ®Google Scholar
• Van Someren, M.W., Y.F. Barnard, and J.A.C. Sandberg. 1994. The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes. London: Academic Press.
   Google Scholar
• Wickens, C.D. 2008. “Multiple Respources and Mental Workload.” Human Factors: The Journal of the Human Factors and Ergonomics Society 50 (3): 449–455. doi:10.1518/001872008X288394.
   PubMed Web of Science ®Google Scholar
• Wickens, C.D., and C.M. Carswell. 2021. “Information Processing.” In Handbook of Human Factors and Ergonomics. 5th ed., edited by G. Salvendy and W. Karwowski. John Wiley and Sons, Inc.
   Google Scholar
• Wilson, M.R., J.M. Poolton, N. Malhotra, K. Ngo, E. Bright, and R.S.W. Masters. 2011. “Development and Validation of a Surgical Workload Measure: The Surgery Task Load Index (SURG-TLX).” World Journal of Surgery 35 (9): 1961–1969. doi:10.1007/s00268-011-1141-4.
   PubMed Web of Science ®Google Scholar
• Woods, D.D., and E. Hollnagel. 2005. Joint Cognitive Systems: Patterns in Cognitive Systems Engineering. Boca Raton, FL: CRC Press. doi:10.1201/9781420005684.
   Google Scholar
• Young, M.S., K.A. Brookhuis, C.D. Wickens, and P.A. Hancock. 2015. “State of Science: Mental Workload in Ergonomics.” Ergonomics 58 (1): 1–17. doi:10.1080/00140139.2014.956151.
   PubMed Web of Science ®Google Scholar