Comparison of NASA-TLX scale, modified Cooper–Harper scale and mean inter-beat interval as measures of pilot mental workload during simulated flight tasks

References

• Aasman, J., G. Mulder, and L. J. Mulder. 1987. “Operator Effort and the Measurement of Heart-Rate Variability.” Human Factors: The Journal of the Human Factors and Ergonomics Society 29 (2): 161–170. doi:10.1177/001872088702900204.
   PubMed Web of Science ®Google Scholar
• Akyeampong, J., S. Udoka, G. Caruso, and M. Bordegoni. 2014. “Evaluation of Hydraulic Excavator Human–Machine Interface Concepts Using NASA-TLX.” International Journal of Industrial Ergonomics 44 (3): 374–382. doi:10.1016/j.ergon.2013.12.002.
   Web of Science ®Google Scholar
• Annett, J. 2002. “Subjective Rating Scales: Science or Art?” Ergonomics 45 (14): 966–987. doi:10.1080/00140130210166951.
   PubMed Web of Science ®Google Scholar
• Battiste, V., and M. Bortolussi. 1988. “Transport Pilot Workload: A Comparison of Two Subjective Techniques.” Proceedings of the Human Factors Society Annual Meeting 32 (2): 150–154. doi:10.1177/154193128803200232.
   Google Scholar
• Byers, J., A. Bittner, and S. Hill. 1989. “Traditional and Raw Task Load Index (TLX) Correlations: Are Paired Comparisons Necessary?” In Advances in Industrial Ergonomics and Safety I, edited by A. Mital, 481–485. London: Taylor and Francis.
   Google Scholar
• Camm, A., M. Malik, J. Bigger, G. Breithardt, S. Cerutti, R. Cohen, P. Coumel, E. Fallen, H. Kennedy, and R. Kleiger. 1996. “Heart Rate Variability: Standards of Measurement, Physiological Interpretation and Clinical Use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology.” Circulation 93 (5): 1043–1065. doi:10.1161/01.CIR.93.5.1043.
   PubMed Web of Science ®Google Scholar
• Carayon, P., A. Hundt, C. Alvarado, S. Springman, and P. Ayoub. 2006. “Patient Safety in Outpatient Surgery: The Viewpoint of the Healthcare Providers.” Ergonomics 49 (5–6): 470–485. doi:10.1080/00140130600568717.
   PubMed Web of Science ®Google Scholar
• Carayon, P., S. Kianfar, Y. Li, A. Xie, B. Alyousef, and A. Wooldridge. 2015. “A Systematic Review of Mixed Methods Research on Human Factors and Ergonomics in Health Care.” Applied Ergonomics 51:291–321. doi:10.1016/j.apergo.2015.06.001.
   PubMed Web of Science ®Google Scholar
• Casali, J., and W. Wierwille. 1983. “A Comparison of Rating Scale, Secondary-Task, Physiological, and Primary-Task Workload Estimation Techniques in a Simulated Flight Task Emphasizing Communications Load.” Human Factors: The Journal of the Human Factors and Ergonomics Society 25 (6): 623–641. doi:10.1177/001872088302500602.
   PubMed Web of Science ®Google Scholar
• Casali, J., and W. Wierwille. 1984. “On the Measurement of Pilot Perceptual Workload: A Comparison of Assessment Techniques Addressing Sensitivity and Intrusion Issues.” Ergonomics 27 (10): 1033–1050. doi:10.1080/00140138408963584.
   PubMed Web of Science ®Google Scholar
• Casner, S., and B. Gore. 2010. Measuring and Evaluating Workload: A Primer. NASA Technical Memorandum (2010-216395), San Jose, CA: Ames Research Center.
   Google Scholar
• ChuDuc, H., K. NguyenPhan, and D. NguyenViet. 2013. “A Review of Heart Rate Variability and Its Applications.” APCBEE Procedia 7:80–85. doi:10.1016/j.apcbee.2013.08.016.
   Google Scholar
• Dahlstrom, N., and S. Nahlinder. 2009. “Mental Workload in Aircraft and Simulator during Basic Civil Aviation Training.” The International Journal of Aviation Psychology 19 (4): 309–325. doi:10.1080/10508410903187547.
   Web of Science ®Google Scholar
• Dekker, S. 2012. “Complexity, Signal Detection, and the Application of Ergonomics: Reflections on a Healthcare Case Study.” Applied Ergonomics 43 (3): 468–472. doi:10.1016/j.apergo.2011.07.003.
   PubMed Web of Science ®Google Scholar
• Dennis, K., and D. Harris. 1998. “Computer-Based Simulation as an Adjunct to Ab Initio Flight Training.” The International Journal of Aviation Psychology 8 (3): 261–276. doi:10.1207/s15327108ijap0803_6.
   Web of Science ®Google Scholar
• Di Nocera, F., M. Camilli, and M. Terenzi. 2007. “A Random Glance at the Flight Deck: Pilots' Scanning Strategies and the Real-Time Assessment of Mental Workload.” Journal of Cognitive Engineering and Decision Making 1 (3): 271–285. doi:10.1518/155534307X255627.
   Google Scholar
• DiDomenico, A., and M. Nussbaum. 2008. “Interactive Effects of Physical and Mental Workload on Subjective Workload Assessment.” International Journal of Industrial Ergonomics 38 (11–12): 977–983. doi:10.1016/j.ergon.2008.01.012.
   Web of Science ®Google Scholar
• Fallahi, M., M. Motamedzade, R. Heidarimoghadam, A. Soltanian, and S. Miyake. 2016. “Effects of Mental Workload on Physiological and Subjective Responses during Traffic Density Monitoring: A Field Study.” Applied Ergonomics 52:95–103. doi:10.1016/j.apergo.2015.07.009.
   PubMed Web of Science ®Google Scholar
• Gabriel, G.,. M. Ramallo, and E. Cervantes. 2016. “Workload Perception in Drone Flight Training Simulators.” Computers in Human Behavior 64:449–454. doi:10.1016/j.chb.2016.07.040.
   Web of Science ®Google Scholar
• Gopher, D., and E. Donchin. 1986. “Workload – an Examination of the Concept.” In Handbook of Perception and Human Performance. Vol 2 of Cognitive Processes and Performance, edited by K. Boff, L. Kaufman, and J. P. Thomas, 1–49. New York, NY: Wiley-Interscience.
   Google Scholar
• Hart, S. 2006. “NASA-Task Load Index (NASA-TLX); 20 Years Later.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50 (9): 904–908. doi:10.1177/154193120605000909.
   Google Scholar
• Hart, S., and L. Staveland. 1988. “Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research.” Advances in Psychology 52:139–183. doi:10.1016/S0166-4115(08)62386-9.
   Google Scholar
• Hill, S., H. Iavecchia, J. Byers, A. Bittner, A. Zaklade, and R. Christ. 1992. “Comparison of Four Subjective Workload Rating Scales.” Human Factors: The Journal of the Human Factors and Ergonomics Society 34 (4): 429–439. doi:10.1177/001872089203400405.
   Web of Science ®Google Scholar
• Hockey, R. 1997. “Compensatory Control in the Regulation of Human Performance under Stress and High Workload: A Cognitive-Energetical Framework.” Biological Psychology 45 (1–3): 73–93. doi:10.1016/S0301-0511(96)05223-4.
   PubMed Web of Science ®Google Scholar
• Johanssen, G., N. Moray, R. Pew, J. Rasmussen, A. Sanders, and C. Wickens. 1979. “Final Report of Experimental Psychology Group.” In Mental Workload Vol. 8 of the NATO Conference Series, edited by N. Moray, 101–114. New York, NY: Springer.
   Google Scholar
• Jorna, P. 1993. “Heart Rate and Workload Variations in Actual and Simulated Flight.” Ergonomics 36 (9): 1043–1054. doi:10.1080/00140139308967976.
   PubMed Web of Science ®Google Scholar
• Kahneman, D. 1973. Attention and Effort. Englewood Cliffs, NJ: Prentice Hall.
   Google Scholar
• Mansikka, H., P. Simola, K. Virtanen, D. Harris, and L. Oksama. 2016. “Fighter Pilots’ Heart Rate, Heart Rate Variation and Performance during Instrument Approaches.” Ergonomics 59 (10): 1344–1352. doi:10.1080/00140139.2015.1136699.
   PubMed Web of Science ®Google Scholar
• Mansikka, H., K. Virtanen, D. Harris, and P. Simola. 2016. “Fighter Pilots’ Heart Rate, Heart Rate Variation and Performance during an Instrument Flight Rules Proficiency Test.” Applied Ergonomics 56:213–219. doi:10.1016/j.apergo.2016.04.006.
   PubMed Web of Science ®Google Scholar
• Michell, J. 1997. “Quantitative Science and the Definition of Measurement in Psychology.” British Journal of Psychology 88 (3): 355–383. doi:10.1111/j.2044-8295.1997.tb02641.x.
   Web of Science ®Google Scholar
• Moroney, W., D. Biers, F. Eggemeier, and J. Mitchell. 1992. “A Comparison of Two Scoring Procedures with the NASA Task Load Index in a Simulated Flight Task.” Proceedings of the IEEE 1992 National, Dayton, OH: IEEE, May 18–22.
   Google Scholar
• Mulder, G. 1986. “The Concept and Measurement of Mental Effort.” In Energetics and Human Information Processing, edited by G. Hockey. Dordrecht: Springer.
   Google Scholar
• Newell, G., and N. Mansfield. 2008. “Evaluation of Reaction Time Performance and Subjective Workload during Whole-Body Vibration Exposure While Seated in Upright and Twisted Postures with and without Armrests.” International Journal of Industrial Ergonomics 38 (5–6): 499–508. doi:10.1016/j.ergon.2007.08.018.
   Web of Science ®Google Scholar
• Norman, D., and D. Bobrow. 1975. “On Data-Limited and Resource-Limited Processes.” Cognitive Psychology 7 (1): 44–64. doi:10.1016/0010-0285(75)90004-3.
   Web of Science ®Google Scholar
• O'Donnell, R., F. Eggemeier, and F. Thomas. 1986. “Workload Assessment Methodology.” In Handbook of Perception and Human Performance. Vol 2 of Cognitive Processes and Performance, edited by K. Boff, L. Kaufman, and J. P. Thomas, 1–49. New York, NY: Wiley-Interscience.
   Google Scholar
• Opmeer, C. 1973. “The Information Content of Successive RR-Interval Times in the ECG. Preliminary Results Using Factor Analysis and Frequency Analysis.” Ergonomics 16 (1): 105–112. doi:10.1080/00140137308924486.
   PubMed Web of Science ®Google Scholar
• Prichard, J., L. Bizo, and R. Stratford. 2011. “Evaluating the Effects of Team-Skills Training on Subjective Workload.” Learning and Instruction 21 (3): 429–440. doi:10.1016/j.learninstruc.2010.06.003.
   Web of Science ®Google Scholar
• Roscoe, A. 1975. “Heart Rate Monitoring of Pilots during Steep-Gradient Approaches.” Aviation, Space, and Environmental Medicine 46 (11): 1410–1413.
   PubMedGoogle Scholar
• Roscoe, A. 1993. “Heart Rate as a Psychophysiological Measure for In-Flight Workload Assessment.” Ergonomics 36 (9): 1055–1062. doi:10.1080/00140139308967977.
   PubMed Web of Science ®Google Scholar
• Sato, N., T. Kamada, S. Miyake, J. Akatsu, M. Kumashiro, and Y. Kume. 1999. “Subjective Mental Workload in Type a Women.” International Journal of Industrial Ergonomics 24 (3): 331–336. doi:10.1016/S0169-8141(98)00060-2.
   Web of Science ®Google Scholar
• Skipper, J., C. Rieger, and W. Wierwille. 1986. “Evaluation of Decision-Tree Rating Scales for Mental Workload Estimation.” Ergonomics 29 (4): 585–599. doi:10.1080/00140138608968293.
   PubMed Web of Science ®Google Scholar
• Svensson, E., M. Angelborg-Thanderez, L. Sjöberg, and S. Olsson. 1997. “Information Complexity-Mental Workload and Performance in Combat Aircraft.” Ergonomics 40 (3): 362–380. doi:10.1080/001401397188206.
   PubMed Web of Science ®Google Scholar
• Svensson, E., M. Angelborg-Thanderz, and G. Wilson. 1999. Models of Pilot Performance for Systems and Mission Evaluation - Psychological and Psychophysiological Aspects. Interim report. Wright-Patterson Air Force Base, Dayton, OH: Human Effectiveness Directorate.
   Google Scholar
• Svensson, E. A. I., and G. F. Wilson. 2002. “Psychological and Psychophysiological Models of Pilot Performance for Systems Development and Mission Evaluation.” The International Journal of Aviation Psychology 12 (1): 95–110. doi:10.1207/S15327108IJAP1201_8.
   Web of Science ®Google Scholar
• van Westrenen, F. 1996. “A Framework for a Decision Model of River-Pilots Based on Their Workload.” International Journal of Industrial Ergonomics 17 (5): 411–418. doi:10.1016/0169-8141(94)00118-9.
   Web of Science ®Google Scholar
• Veltman, J., and A. Gaillard. 1998. “Physiological Workload Reactions to Increasing Levels of Task Difficulty.” Ergonomics 41 (5): 656–669. doi:10.1080/001401398186829.
   PubMed Web of Science ®Google Scholar
• Verwey, W., and H. Veltman. 1996. “Detecting Short Periods of Elevated Workload: A Comparison of Nine Workload Assessment Techniques.” Journal of Experimental Psychology: Applied 2 (3): 270–285. doi:10.1037/1076-898X.2.3.270.
   Web of Science ®Google Scholar
• Vicente, K., D. Thornton, and N. Moray. 1987. “Spectral Analysis of Sinus Arrhythmia: A Measure of Mental Effort.” Human Factors: The Journal of the Human Factors and Ergonomics Society 29 (2): 171–182. doi:10.1177/001872088702900205.
   PubMed Web of Science ®Google Scholar
• Wang, L., X. He, and Y. Chen. 2016. “Quantitative Relationship Model between Workload and Time Pressure under Different Flight Operation Tasks.” International Journal of Industrial Ergonomics 54: 93–102. doi:10.1016/j.ergon.2016.05.008.
   Web of Science ®Google Scholar
• Wickens, C. 1991. “Processing Resources and Attention.” In Multiple-Task Performance, edited by D. Damos, 3–34. London: Taylor and Francis.
   Google Scholar
• Wickens, C. 2008. “Multiple Resources and Mental Workload.” Human Factors: The Journal of the Human Factors and Ergonomics Society 50 (3): 449–455. doi:10.1518/001872008X288394.
   PubMed Web of Science ®Google Scholar
• Wierwille, W., and J. Casali. 1983. “A Validated Rating Scale for Global Mental Workload Measurement Applications.” Proceedings of the Human Factors Society Annual Meeting 27 (2): 129–133. doi:10.1177/154193128302700203.
   Google Scholar
• Wierwille, W., and F. Eggemeier. 1993. “Recommendations for Mental Workload Measurement in a Test and Evaluation Environment.” Human Factors: The Journal of the Human Factors and Ergonomics Society 35 (2): 263–281. doi:10.1177/001872089303500205.
   Web of Science ®Google Scholar
• Wierwille, W.,. M. Rahimi, and J. Casali. 1985. “Evaluation of 16 Measures of Mental Workload Using a Simulated Flight Task Emphasizing Mediational Activity.” Human Factors: The Journal of the Human Factors and Ergonomics Society 27 (5): 489–502. doi:10.1177/001872088502700501.
   Web of Science ®Google Scholar
• Williges, R., and W. Wierwille. 1979. “Behavioral Measures of Aircrew Mental Workload.” Human Factors: The Journal of the Human Factors and Ergonomics Society 21 (5): 549–574. doi:10.1177/001872087902100503.
   Web of Science ®Google Scholar
• Wilson, G. 1993. “Air-to-Ground Training Missions: A Psychophysiological Workload Analysis.” Ergonomics 36 (9): 1071–1087. doi:10.1080/00140139308967979.
   PubMed Web of Science ®Google Scholar
• Ylönen, H., H. Lyytinen, T. Leino, J. Leppäluoto, and P. Kuronen. 1997. “Heart Rate Responses to Real and Simulated BA Hawk MK 51 Flight.” Aviation, Space, and Environmental Medicine 68 (7): 601–605.
   PubMedGoogle Scholar
• Zhang, Y., Z. Li, B. Wu, and S. Wu. 2009. “A Spaceflight Operation Complexity Measure and Its Experimental Validation.” International Journal of Industrial Ergonomics 39 (5): 756–765. doi:10.1016/j.ergon.2009.03.003.
   Web of Science ®Google Scholar