A scalable Bayesian framework for large-scale sensor-driven network anomaly detection

Abstract

Many real systems have a network/graph structure with many connected nodes and many edges representing deterministic or stochastic dependencies and interactions between nodes. Various types of known or unknown anomalies and disturbances may occur across these networks over time. Developing real-time anomaly detection and isolation frameworks is crucial to enable network operators to make more informed and timely decisions and take appropriate maintenance and operations actions. To monitor the health of modern networks in real time, different types of sensors and smart devices are installed across these networks that can track real-time data from a particular node or a section of a network. In this article, we introduce an innovative inference method to calculate the most probable explanation of a set of hidden nodes in heterogeneous attributed networks with a directed acyclic graph structure represented by a Bayesian network, given the values of a set of binary data observed from available sensors, which may be located only at a subset of nodes. The innovative use of Bayesian networks to incorporate parallelization and vectorization makes the proposed framework applicable for large-scale graph structures. The efficiency of the model is shown through a comprehensive set of numerical experiments.

KEYWORDS:

• Anomaly detection
• Bayesian networks
• sensor analytics
• system monitoring

Additional information

Funding

This article is based on work supported by National Science Foundation under Grant No. 1846975.

Notes on contributors

Feiran Xu

Feiran Xu is a doctoral student in Industrial and Systems Engineering at the University of Miami. Her research is focused on anomaly detection in large-scale sensor-driven networks. She holds a Master of Science in Global Logistics from W.P. Carey School of Business at Arizona State University and a Bachelor of Management in Management Science from Sichuan University in Chengdu, China. Currently, she is a Graduate Teaching Assistant in the department and has presented her research at several conferences.

Ramin Moghaddass

Ramin Moghaddass is an Associate Professor in the College of Engineering’s Department of Industrial & Systems Engineering and the Director of the DOE Industrial Assessment Center and Data Analytics Lab. Dr. Moghaddass studies complex, sensor-driven engineered systems. His research is on developing a new generation of flexible and large-scale models for real-time system health monitoring inspired by dynamic structures and networks. His research is applicable in a number of engineering systems, including wind turbines, power systems, and other sensor-intensive engineering systems.