Multi-view crowd congestion monitoring system based on an ensemble of convolutional neural network classifiers

Abstract

Multi-view video surveillance is a highly valuable tool to ensure the safety of the crowd in large public space. By utilizing complementary information captured by multiple cameras, the issue of limited views and occlusion in single views can be addressed to gain better insight into the whole monitored space. However, multi-view surveillance has been widely applied to microscopic crowd analysis, for example pedestrian detection and tracking, while macroscopic level analysis, which deals with the whole crowd, has received little attention. We propose a multi-view framework for the generation of level of service maps, which are the most commonly used measure of congestion at macroscopic level, based on an ensemble of state-of-the-art Convolutional Neural Networks (CNNs). Several combination rules are compared and evaluated on two datasets, both in sparse and dense scenarios. Our results show that this fusion framework improves the accuracy of level of service map generation, from 83.2% to 89.8%, and eliminates blind spots in single views. Our framework is implemented on a 3 D GIS platform, which provides a suitable interface for multi-view crowd congestion management. The results of a loading test show that a maximum of 48 cameras can be processed at a map refresh rate of 2 seconds.

Keywords:

• 3D GIS
• convolutional neural networks
• crowd congestion
• ensemble learning
• level of service
• multi-view video surveillance

Acknowledgements

The authors would like to thank Mr. Abdullah Alhawsawi for providing the CCTV videos for the experiment.