Statistical Process Monitoring of Artificial Neural Networks

Abstract

The rapid advancement of models based on artificial intelligence demands innovative monitoring techniques which can operate in real time with low computational costs. In machine learning, especially if we consider artificial neural networks (ANNs), the models are often trained in a supervised manner. Consequently, the learned relationship between the input and the output must remain valid during the model’s deployment. If this stationarity assumption holds, we can conclude that the ANN provides accurate predictions. Otherwise, the retraining or rebuilding of the model is required. We propose considering the latent feature representation of the data (called “embedding”) generated by the ANN to determine the time when the data stream starts being nonstationary. In particular, we monitor embeddings by applying multivariate control charts based on the data depth calculation and normalized ranks. The performance of the introduced method is compared with benchmark approaches for various ANN architectures and different underlying data formats.

Keyword:

• Artificial neural networks
• Change point detection
• Data depth
• Latent feature representation
• Multivariate statistical process monitoring
• Online process monitoring

Acknowledgments

The authors thank William H. Woodall for his valuable comments and suggestions. We thank the editors and the two reviewers for their thorough reading and helpful suggestions. Furthermore, we appreciated the fruitful discussions during the seminars at the Helmut Schmidt University Hamburg and the Leibniz University Hannover.

Disclosure Statement

The authors report there are no competing interests to declare.

Notes

1 https://www.cs.toronto.edu/∼kriz/cifar.html

Additional information

Funding

We acknowledge the support of the cluster system team at the Leibniz University Hannover, Germany, in the production of this work. The project is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - 412992257.