Analysing operators' diagnostic reasoning during multiple events

Abstract

This field study in the process control industry adopts a process tracing approach to investigate difficulties in diagnostic reasoning experienced by operations personnel when confronted with multiple fault scenarios. Three fault scenarios (two multiple and one single) are each tackled by three senior operations personnel on a high-fidelity simulator. Verbal reports are collected together with a video recording of actions and accompanying plant states. While a single fault is diagnosed successfully on two of the three occasions, neither multiple fault is diagnosed correctly. A process tracing approach is developed to investigate the nature of the difficulties in diagnostic reasoning. This involves: transcription of verbalizations, actions, etc. onto a time line; initial encoding of transcripts using 10 behaviourally-related categories that are further subdivided; and development of a technique named MAPS (Mental States and Activities in the Problem Space) from Newell and Simon's problem behaviour graph, which provides a visuospatial representation of the progression through knowledge states and activities during problem solving. Difficulties principally occur in the hypothesis generation stages of diagnostic reasoning. It is speculated that this is due to the development of a restrictive habitual assumption that there is a single fault. It is concluded that the boundaries that habit may impose on hypothesis generation in process control contexts be further investigated together with the development of training procedures to reduce such negative effects.