Modeling Recurrent Failures on Large Directed Networks

Abstract

Many lifeline infrastructure systems consist of thousands of components configured in a complex directed network. Disruption of the infrastructure constitutes a recurrent failure process over a directed network. Statistical inference for such network recurrence data is challenging because of the large number of nodes with irregular connections among them. Motivated by 16 years of Scottish Water operation records, we propose a network Gamma-Poisson Autoregressive NHPP (GPAN) model for recurrent failure data from large-scale directed physical networks. The model consists of two layers: the temporal layer applies a Non-Homogeneous Poisson Process (NHPP) with node-specific frailties, and the spatial layer uses a well-orchestrated gamma-Poisson autoregressive scheme to establish correlations among the frailties. Under the network-GPAN model, we develop a sum-product algorithm to compute the marginal distribution for each frailty conditional on the recurrence data. The marginal conditional frailty distributions are useful for predicting future failures based on historical data. In addition, the ability to rapidly compute these marginal distributions allows adoption of an EM type algorithm for estimation. Through a Bethe approximation, the output from the sum-product algorithm is used to compute maximum log-likelihood estimates. Applying the methods to the Scottish Water network, we demonstrate utility in aiding operation management and risk assessment of the water utility. Supplementary materials for this article are available online including a standardized description of the materials available for reproducing the work.

KEYWORDS:

• Bethe approximation
• Dynamic network data
• Gamma-Poisson autoregression
• Sum-product algorithm
• Risk assessment

Supplementary Materials

Supplementary Material for “Modeling Recurrent Failures on Large Directed Networks”: Technical details of non-homogeneous Poisson processes; detailed derivations of the forward and backward messages in Section 3; proof of Theorem 1; detailed derivation of the Bethe approximation in Section 4.2; computation time for the simulation studies in Section 5 and the real data analysis in Section 6; numerical comparison with the Monte Carlo EM algorithm; numerical comparison of network-GPAN model with the NHPP model without frailty and the Poisson model with log-normal frailty on the Scottish Water data; details of robust standard errors for the estimators of model parameters.

Code.zip: Matlab codes for all the simulation studies and the real data analysis.

Acknowledgments

The authors thank the Editor, the Associate Editor and the anonymous reviewers for their valuable comments and suggestions that have helped improve the article.

Disclosure Statement

The authors report there are no competing interests to declare.

Additional information

Funding

Qingqing Zhai was partially supported by the National Natural Science Foundation of China (72271154, 71901138). Zhisheng Ye and Cheng Li were supported by Singapore Ministry of Education Academic Research Funds Tier 2 grant MOE2019-T2-1-177. David Dunson was partially supported by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 856506), United States National Institutes of Health (R01ES028804, R01ES035625), and Office of Naval Research (N00014-21-1-2510).