Predicting the false alarm rate in multi-institution mortality monitoring

Abstract

Statistical process control is increasingly used by single hospitals or centres to monitor their performance, but national monitoring across multiple centres, measures and groups incurs higher false alarm rates unless the method is modified. We consider setting the threshold for cumulative sum charts to produce the desired false alarm rate, taking into account the centre volume and expected outcome rate. We used simulation to estimate the false alarm and successful detection rates for a variety of chart thresholds. We thereby calculated the ‘cost’ of a higher threshold compared with one set to give a false alarm rate of 5% for three clinical groups of common interest. The false alarm rate often showed non-linear relations with the threshold, volume and expected mortality rate but an equation was found with good approximation to the simulated values. The relation between these factors and the ‘cost’ of a higher threshold was not straightforward. The ‘cost’ (difference in number of deaths) incurred by raising the chart threshold provides an intuitive measure and is applicable to other settings.

Keywords:

• quality
• statistics
• simulation
• methodology
• health service