Decision-making and response strategies in interaction with alarms: the impact of alarm reliability, availability of alarm validity information and workload

Abstract

Responding to alarm systems which usually commit a number of false alarms and/or misses involves decision-making under uncertainty. Four laboratory experiments including a total of 256 participants were conducted to gain comprehensive insight into humans' dealing with this uncertainty. Specifically, it was investigated how responses to alarms/non-alarms are affected by the predictive validities of these events, and to what extent response strategies depend on whether or not the validity of alarms/non-alarms can be cross-checked against other data. Among others, the results suggest that, without cross-check possibility (experiment 1), low levels of predictive validity of alarms ( ≤ 0.5) led most participants to use one of two different strategies which both involved non-responding to a significant number of alarms (cry-wolf effect). Yet, providing access to alarm validity information reduced this effect dramatically (experiment 2). This latter result emerged independent of the effort needed for cross-checkings of alarms (experiment 3), but was affected by the workload imposed by concurrent tasks (experiment 4). Theoretical and practical consequences of these results for decision-making and response selection in interaction with alarm systems, as well as the design of effective alarm systems, are discussed.

Abstract

Practitioner Summary: Four laboratory experiments were performed to investigate the effects of false alarms and misses on the behavioural effectiveness of alarm systems. The results provide insight in determinants of compliance with alarms, dependent on the alarm system's reliability, the possibility to cross-check the validity of alarms and the workload imposed by concurrent tasks.

Keywords::

• decision-making
• alarm system
• compliance
• reliance
• cry-wolf effect

Acknowledgements

Thanks are due to Marcus Bleil for programming the M-TOPS-A software, and Marlene Vogel, Ulrike Schmuntzsch and Julia Zorn for their support with data collection and analysis. We further thank Magali Balaud and three anonymous reviewers for their helpful comments to earlier versions of this article.

Notes

^1. Note that alarm systems with PPVs of 0.1 and 0.9, albeit representing very different systems in terms of reliability are equivalent with respect to the level of uncertainty associated with alarms. Humans interacting with these systems can be sure that 90% of alarms are either wrong or correct, respectively, i.e. they face the same low uncertainty in making their decision how to respond.

Additional information

Funding

The research presented in this article was supported by a scholarship provided to the second author by the German Science Foundation (DFG).