Observational assessment of situation awareness, team differences and training implications

Abstract

The two goals were to investigate, first, the practicability and reliability of observational assessment of team situation awareness (SA) and, second, the nature of any team differences, their consistency and training implications. Five shift teams tackled three scenarios, each with three probe events concerning SA, and three observers viewed and rated concurrently each shift. This methodology was found to be practicable and achieved satisfactory rater reliability as indicated by intraclass and inter-rater correlations. Team differences in SA emerged although there was no consistent pattern. A retrospective analysis of individual and team behaviours relating to SA was performed using the Critical Incident Technique. A total of 75 incidents and 20 behavioural dimensions relevant to SA were identified and these were subsumed under planning, problem solving, team coordination, attention, communication and knowledge. These findings are discussed with regard to the nature and measurement of SA, and the content of training to improve SA for control room teams.

Keywords:

• Team situation awareness
• Observational method
• Rating reliability
• Training

Acknowledgements

The authors gratefully acknowledge financial support for this project from the Industry Management Committee of the UK HSC Coordinated Programme of Nuclear Safety Research. Thanks are due to Ian Umbers and Steve Belton, British Energy, for comments on an earlier draft of this paper and also to two anonymous referees who provided constructive criticisms.