Empirical evaluation of the Process Overview Measure for assessing situation awareness in process plants

References

• Bennett, Kevin B., Allen L. Nagy, and John M. Flach. 2006. “Visual Displays.” In Handbook of Human Factors and Ergonomics, edited by Gavriel Salvendy, 1191–1219. Hoboken, NJ: John Wiley & Sons.
   Google Scholar
• Bennett, Kevin B., Mona L. Toms, and David D. Woods. 1993. “Emergent Features and Graphical Elements: Designing More Effective Configural Displays.” Human Factors 35: 71–97.
   Web of Science ®Google Scholar
• Burns, Catherine M., Gyrd Skraaning, Jr., Greg A. Jamieson, Nathan Lau, Jordanna Kwok, Robin Welch, and Gisle Andresen. 2008. “Evaluation of Ecological Interface Design for Nuclear Process Control: Situation Awareness Effects.” Human Factors: The Journal of the Human Factors and Ergonomics Society 50: 663–679.10.1518/001872008X312305
   PubMed Web of Science ®Google Scholar
• Burns, Catherine M., and John R. Hajdukiewicz. 2004. Ecological Interface Design. Boca Raton, FL: CRC Press.
   Google Scholar
• Cohen, Jacob. 1960. “A Coefficient of Agreement for Nominal Scales.” Educational and Psychological Measurement 20 (1): 37–46.10.1177/001316446002000104
   Web of Science ®Google Scholar
• Cooper, Simon, Joanne Porter, and Linda Peach. 2014. “Measuring Situation Awareness in Emergency Settings: A Systematic Review of Tools and Outcomes.” Open Access Emergency Medicine 6: 1–7.
   PubMedGoogle Scholar
• Durso, Francis T., and Frank A. Drews. 2010. “Health Care, Aviation, and Ecosystems: A Socio-natural Systems Perspective.” Current Directions in Psychological Science 19 (2): 71–75. doi:10.1177/0963721410364728.
   Web of Science ®Google Scholar
• Eitrheim, Maren H. R., Gyrd Skraaning, Jr., Nathan Lau, Tommy Karlsson, Christer Nihlwing, Mario Hoffmann, and Jan Erik Farbrot. 2010. Staffing Strategies in Highly Automated Future Plants: Results from the 2009 HAMMLAB Experiment, HWR-938. Halden: OECD Halden Reactor Project.
   Google Scholar
• Endsley, Mica R. 1995a. “Measurement of Situation Awareness in Dynamic Systems.” Human Factors: The Journal of the Human Factors and Ergonomics Society 37 (1): 65–84.10.1518/001872095779049499
   Web of Science ®Google Scholar
• Endsley, Mica R. 1995b. “Toward a Theory of Situation Awareness in Dynamic Systems.” Human Factors: The Journal of the Human Factors and Ergonomics Society 37 (1): 32–64.10.1518/001872095779049543
   Web of Science ®Google Scholar
• Hallbert, Bruce P. 1997. “Situation Awareness and Operator Performance: Results from Simulator-based Studies.” IEEE 6th Conference on Human Factors and Power Plants, Orlando, FL, USA.
   Google Scholar
• Hignett, Sue. 2014. “Commentary on the Paper by Heimrich Kanis Entitled: Reliability and Validity of Findings in Ergonomics Research.” Theoretical Issues in Ergonomics Science 15 (1): 50–54. doi:10.1080/1463922X.2013.778354.
   Google Scholar
• Hogg, David N., Knut Follesø, Frode Strand-Volden, and Belén Torralba. 1995. “Development of a Situation Awareness Measure to Evaluate Advanced Alarm Systems in Nuclear Power Plant Control Rooms.” Ergonomics 38 (11): 2394–2413.10.1080/00140139508925275
   Web of Science ®Google Scholar
• Howell, David. 2002. Statistical Methods for Psychology. 5th ed. Pacific Grove, CA: Duxbury/Thomson Learning.
   Google Scholar
• Hurlen, Lars, Christer Nihlwing, Arild Teigen, and H. Jokstad. 2010. Trasparent Automation Displays – Design and Evaluation, HWR-935. Halden: OECD Halden Reactor Project.
   Google Scholar
• Jarmasz, J., and J. G. Hollands. 2009. “Confidence intervals in repeated-measures designs: The number of observations principle.” Canadian Journal of Experimental Psychology 63 (2): 124–138. doi:10.1037/a0014164.
   PubMed Web of Science ®Google Scholar
• Jones, Debra G., and M. R. Endsley. 2004. “Use of Real-time Probes for Measuring Situation Awareness.” The International Journal of Aviation Psychology 14 (4): 343–367.10.1207/s15327108ijap1404_2
   Web of Science ®Google Scholar
• Kanis, H. 2014. “Reliability and Validity of Findings in Ergonomics Research.” Theoretical Issues in Ergonomics Science 15 (1): 1–46. doi:10.1080/1463922X.2013.802058.
   Google Scholar
• Kanis, H., J. P. L. Schoormans, and W. S. Green. 2014. “Reliability and Validity Revisited.” Theoretical Issues in Ergonomics Science 15 (1): 62–68. doi:10.1080/1463922X.2013.803170.
   Google Scholar
• Karlsson, T., H. Jokstad, B. D. Meyer, C. Nilhlwing, S. Norrman, E. K. Puska, P. Raussi, and O. Tiihonen. 2001. OECD Halden Reactor Project: The HAMBO BWR Simulator of HAMMLAB, HWR-663. Halden: OECD Halden Reactor Project.
   Google Scholar
• Kraemer, Helena Chmura. 1992. “How Many raters? Toward the Most Reliable Diagnostic Consensus.” Statistics in Medicine 11 (3): 317–331. doi:10.1002/sim.4780110305.
   PubMed Web of Science ®Google Scholar
• Lang, Andrew W., Emilie M. Roth, Kent Bladh, and Roger Hine. 2002. “Using a Benchmark-referenced Approach for Validating a Power Plant Control Room: Results of the Baseline Study.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46: 1878–1882.10.1177/154193120204602302
   Google Scholar
• Lau, Nathan. 2012. “Situation Awareness in Process Control and the Process Overview Measure.” Unpublished Doctoral Dissertation, Dept. of Mechanical and Industrial Engineering, University of Toronto.
   Google Scholar
• Lau, Nathan, Greg A. Jamieson, Gyrd Skraaning, Jr., and Catherine M. Burns. 2008. “Ecological Interface Design in the Nuclear Domain: An Empirical Evaluation of Ecological Displays for the Secondary Subsystems of a Boiling Water Reactor Plant Simulator.” IEEE Transactions on Nuclear Science 55 (6): 3597–3610.10.1109/TNS.2008.2005725
   Web of Science ®Google Scholar
• Lau, Nathan, and Gyrd Skraaning, Jr. 2015. “Exploring Sub-dimensions of Situation Awareness to Support Integrated System Validation.” Proceedings of the 9th American Nuclear Society International Topical Meeting on Nuclear Plant Instrumentation & Control and Human-machine Interface Technologies (NPIC & HMIT): 1071–1081. Charlotte, NC, USA.
   Google Scholar
• Lau, Nathan, Greg A. Jamieson, and Gyrd Skraaning, Jr. 2012. “Situation Awareness in Process Control: A Fresh Look.” Proceedings of the 8th American Nuclear Society International Topical Meeting on Nuclear Plant Instrumentation & Control and Human-Machine Interface Technologies (NPIC & HMIT): 1511–1523. San Diego, CA, USA.
   Google Scholar
• Lau, Nathan, Greg A. Jamieson, and Gyrd Skraaning, Jr. 2013. “Distinguishing Three Accounts of Situation Awareness based on their Domains of Origin.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting: 220–224.10.1177/1541931213571049
   Google Scholar
• Lau, Nathan, Greg A. Jamieson, and Gyrd Skraaning, Jr. 2014. “Inter-rater Reliability of Query/Probe-based Techniques for Measuring Situation Awareness.” Ergonomics 57 (7): 959–972.10.1080/00140139.2014.910612
   PubMed Web of Science ®Google Scholar
• Lau, Nathan, Gyrd Skraaning, Jr., and Greg A. Jamieson. 2015. Situation Awareness in the Diagnosis of Process Disturbances – The Scenario Understanding Concept and Measures, HWR-1132. Halden: OECD Halden Reactor Project.
   Google Scholar
• Lau, Nathan, Greg A. Jamieson, and Gyrd Skraaning, Jr. in review. “Situation Awareness Acquired from Monitoring Process Plants – The Process Overview Concept and Measure.” Ergonomics.
   Google Scholar
• Lau, Nathan, Gyrd Skraaning, Jr., and Greg A. Jamieson. 2009. “Metacognition in Nuclear Process Control.” Proc. of the 17th Triennial World Congress on Ergonomics, Beijing, China, August 9–14, 2009.
   Google Scholar
• Lau, Nathan, Gyrd Skraaning, Jr., Greg A. Jamieson, and Catherine M. Burns. 2008. The Impact of Ecological Displays on Operator Task Performance and Workload, HWR-888. Halden: OECD Halden Reactor Project.
   Google Scholar
• Lau, Nathan, Gyrd Skraaning, Jr., Maren H. R. Eitrheim, Tommy Karlsson, Christer Nihlwing, and Greg A. Jamieson. 2011. The Process Overview Measure: Methodological Developments to Enhance Inter-rater Reliability, HWR-971. Halden: OECD Halden Reactor Project.
   Google Scholar
• Lau, Nathan, Øystein Veland, Jordanna Kwok, Greg A. Jamieson, Catherine M. Burns, Alf Ove Braseth, and Robin Welch. 2008. “Ecological Interface Design in the Nuclear Domain: An Application to the Secondary Subsystems of a Boiling Water Reactor Plant Simulator.” IEEE Transactions on Nuclear Science 55 (6): 3579–3596.10.1109/TNS.2008.2005979
   Web of Science ®Google Scholar
• O’Brien, K. S., and D. O’Hare. 2007. “Situational Awareness Ability and Cognitive Skills Training in a Complex Real-world Task.” Ergonomics 50 (7): 1064–1091.10.1080/00140130701276640
   PubMed Web of Science ®Google Scholar
• O’Hara, J. M., J. C. Higgins, S. A. Fledger, and P. A. Pieringer. 2012. Human Factors Engineering Program Review Model, NUREG-0711. Washington, DC: U.S. Nuclear Regulatory Commission.
   Google Scholar
• Sebok, Angelia. 2000. “Team Performance in Process Control: Influences of Interface Design and Staffing Levels.” Ergonomics 43 (8): 1210–1236.10.1080/00140130050084950
   PubMed Web of Science ®Google Scholar
• Skraaning, Jr., Gyrd. 1998. The Operator Performance Assessment System (OPAS). Halden: OECD Halden Reactor Project.
   Google Scholar
• Skraaning Gyrd, Jr. 2003. Experimental Control Versus Realism: Methodological Solutions for Simulator Studies in Complex Operating Environments, HPR-361. Halden: OECD Halden Reactor Project.
   Google Scholar
• Skraaning Gyrd, Jr., Nathan Lau, Robin Welch, Christer Nihlwing, Gisle Andresen, Liv Hanne Brevig, Øystein Veland, Greg A. Jamieson, Catherine M. Burns, and Jordanna Kwok. 2007. The Ecological Interface Design Experiment (2005), HWR-833. Halden: OECD Halden Reactor Project.
   Google Scholar
• Skraaning Gyrd, Jr., Maren H. R. Eitrheim, Nathan Lau, Christer Nihlwing, Lars Hurlen, and Tommy Karlsson. 2010. Coping with Automation in Future Plants: Results from the 2009 HAMMLAB Experiment, HWR-937. Halden: OECD Halden Reactor Project.
   Google Scholar
• Stanton, Neville A. 2014. “Commentary on the Paper by Heimrich Kanis Entitled ‘Reliability and Validity of Findings In Ergonomics Research’: Where is the Methodology in Ergonomics Methods?” Theoretical Issues in Ergonomics Science 15 (1): 55–61. doi:10.1080/1463922X.2013.778355.
   Google Scholar
• Stanton, Neville A., and Mark S. Young. 1999. “What Price Ergonomics?” Nature 399: 197–198.10.1038/20298
   Web of Science ®Google Scholar
• Stanton, Neville A., and Mark S. Young. 2003. “Giving Ergonomics Away? The Application of Ergonomics Methods by Novices.” Applied Ergonomics 34 (5): 479–490. doi:10.1016/S0003-6870(03)00067-X.
   PubMed Web of Science ®Google Scholar
• Tharanathan, Anand, Jason Laberge, Peter Bullemer, Dal Vernon Reising, and Rich McLain. 2010. “Functional versus Schematic Overview Displays: Impact on Operator Situation Awareness in Process Monitoring.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 54 (4): 319–323. doi:10.1177/154193121005400411.
   Google Scholar
• Vicente, Kim J. 2002. “Ecological Interface Design: Progress and Challenges.” Human Factors: The Journal of the Human Factors and Ergonomics Society 44 (1): 62–78.10.1518/0018720024494829
   PubMed Web of Science ®Google Scholar
• Vicente, Kim J., and Jens Rasmussen. 1992. “Ecological Interface Design: Theoretical Foundations.” IEEE Transactions on Systems, Man, and Cybernetics 22 (4):589–606.10.1109/21.156574
   Web of Science ®Google Scholar
• Vink, Peter. 2014. “Reaction on the Paper ‘Reliability and Validity of Findings in Ergonomics Research’ of Kanis (2013).” Theoretical Issues in Ergonomics Science 15 (1): 47–49. doi:10.1080/1463922X.2013.778353.
   Google Scholar