Developing expertise in military communications planning: do verbal reports change with experience?

Abstract

The purpose of this study was to investigate novices learning to use a mission planning system. Novice participants received one training session, followed by three test sessions. This was compared to expert performance. During the test sessions, all participants were required to ‘think aloud’, based on Ericsson and Simon's (Ericsson, K.A. and Simon, H.A., 1993. Protocol analysis: verbal reports as data. Revised ed. Cambridge, MA: The MIT Press) description of the technique. Objective measures of time to complete the task, number of errors made, mouse travel and number of mouse clicks were also recorded. It was hypothesised that error, time and mouse data would reflect the acquisition of skill and improved problem-solving abilities. The verbal data were analysed in an exploratory fashion to see what changes individuals experienced. The results of the study showed that participants acquired a level of skill across sessions, and that the cognitive processes reflected in the verbal protocols changed as a function of this skill acquisition. These changes in cognitive processing mirrored a number of the differences between novices and experts.

Keywords:

• expertise
• think aloud
• skill acquisition

Acknowledgements

This research from the Human Factors Integration Defence Technology Centre was part-funded by the Human Sciences Domain of the UK Ministry of Defence Scientific Research Programme. Any views expressed are those of the author(s) and do not necessarily represent those of the Ministry of Defence or any other UK government department.