Log-linear stochastic block modeling and monitoring of directed sparse weighted network systems

Abstract

Networks have been widely employed to reflect the relationships of entities in complex systems. In a weighted network, each node corresponds to one entity while the edge weight between two nodes can represent the number of interactions between two associated entities. More and more schemes have been established to monitor the networks, which help identify the possible changes or anomalies in corresponding systems. However, limited works can comprehensively reflect the community structure, node heterogeneity, interaction sparsity and direction of weighted networks in the literature. This article proposes a log-linear stochastic block model with latent features of nodes based on the mixture of Bernoulli distribution and Poisson distribution to characterize the sparse directional interaction counts within network systems. Explicit matrices and vectors are designed to incorporate community structure and enable straightforward maximum likelihood estimation of parameters. We further construct a monitoring statistic based on the generalized likelihood ratio test for change detection of sparse weighted networks. Comparative studies based on simulations and real data are conducted to validate the high efficiency of proposed model and monitoring scheme.

Keywords:

• Directional networks
• zero-inflated Poisson model
• change detection
• statistical process control

Data availability statement

The data supporting the findings of this study are openly available at http://www.cs.cmu.edu/ enron/.

Acknowledgments

The authors would like to thank the editors and referees for their many constructive and insightful comments, which have promoted significant improvements of this article.

Additional information

Funding

The work described in this paper was supported by National Natural Science Foundation of China (No.72002220 and No.72032005) and by Research Grant Council of Hong Kong (No.11203519). It is also funded by Hong Kong Innovation and Technology Commission (InnoHK Project CIMDA) and by the International Science and Technology Cooperation Program of Guangdong Province (Project #2022A0505050047).

Notes on contributors

Junjie Wang

Junjie Wang is an associate professor with School of Business Administration, Zhongnan University of Economics and Law. He received a Bachelor's degree in engineering management from China University of Geosciences, Wuhan. He obtained two PhD degrees respectively in systems engineering and engineering management from City University of Hong Kong and in management science and engineering from Xi'an Jiaotong University in 2018 under the joint PhD program. His research interests include statistical process control and big data analytics.

Ahmed Maged

Ahmed Maged is an assistant professor at the Department of Mechanical Engineering Benha University, Egypt. He received his PhD in 2022 from the City University of Hong Kong in Advanced Design and Systems Engineering. Ahmed's research interest is focused on Quality Engineering, Machine Learning, and Lean Six Sigma.

Min Xie

Min Xie received a PhD degree in quality technology from Linkoping University, Linkoping, Sweden, in 1987. He is currently a Chair Professor with Department of Advanced Design and Systems Engineering, City University of Hong Kong, Kowloon, Hong Kong. He has authored or coauthored about 300 peer-reviewed journal papers and eight books on quality and reliability engineering. His research interests include reliability engineering, quality management, software reliability, and applied statistics. Prof. Xie was elected as a member of the European Academy of Sciences and Arts in 2022. He was the recipient of the prestigious Lee Kuan Yew (LKY) Research Fellowship in 1991. He has chaired many international conferences and given keynote speeches. He also serves as an editor and associate editor and on the editorial board of many established international journals.