Anomaly Detection for a Large Number of Streams: A Permutation-Based Higher Criticism Approach

References

• Aldosari, S. A., and Moura, J. M. F. (2004), “Detection in Decentralized Sensor Networks,” in 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (Vol. 2), pp. ii–277.
   Google Scholar
• Arias-Castro, E., Candès, E. J., and Durand, A. (2011), “Detection of an Anomalous Cluster in a Network,” Annals of Statistics, 39, 278–304.
   Web of Science ®Google Scholar
• Arias-Castro, E., Candès, E. J., and Plan, Y. (2011), “Global Testing under Sparse Alternatives: Anova, Multiple Comparisons and the Higher Criticism,” Annals of Statistics, 39, 2533–2556.
   Web of Science ®Google Scholar
• Arias-Castro, E., Castro, R. M., Tánczos, E., and Wang, M. (2018), “Distribution-Free Detection of Structured Anomalies: Permutation and Rank-based Scans,” Journal of the American Statistical Association, 113, 789–801. DOI: 10.1080/01621459.2017.1286240.
   Web of Science ®Google Scholar
• Arias-Castro, E., and Wang, M. (2017), “Distribution-Free Tests for Sparse Heterogeneous Mixtures,” TEST, 26, 71–94. DOI: 10.1007/s11749-016-0499-x.
   Web of Science ®Google Scholar
• Benjamini, Y., and Hochberg, Y. (1995), “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing,” Journal of the Royal Statistical Society, Series B, 57, 289–300. DOI: 10.1111/j.2517-6161.1995.tb02031.x.
   Web of Science ®Google Scholar
• Benjamini, Y., and Yekutieli, D. (2001), “The Control of the False Discovery Rate in Multiple Testing under Dependency,” The Annals of Statistics, 29, 1165–1188. DOI: 10.1214/aos/1013699998.
   Web of Science ®Google Scholar
• Berk, R. H., and Jones, D. H. (1979), “Goodness-of-Fit Test Statistics that Dominate the Kolmogorov Statistics,” Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 47, 47–59. DOI: 10.1007/BF00533250.
   Web of Science ®Google Scholar
• Delaigle, A., and Hall, P. (2009), “Higher Criticism in the Context of Unknown Distribution, Non-independence and Classification,” in Perspectives in Mathematical Sciences I: Probability and Statistics, eds. N. S. Narasimha Sastry, T. S. S. R. K. Rao, M. Delampady, and B. Rajeev, pp. 109–138, Singapore: World Scientific.
   Google Scholar
• Delaigle, A., Hall, P., and Jin, J. (2011), “Robustness and Accuracy of Methods for High Dimensional Data Analysis based on Student’s t-statistic,” Journal of the Royal Statistical Society, Series B, 73, 283–301. DOI: 10.1111/j.1467-9868.2010.00761.x.
   Google Scholar
• Donoho, D., and Jin, J. (2004), “Higher Criticism for Detecting Sparse Heterogeneous Mixtures,” Annals of Statistics, 32, 962–994.
   Web of Science ®Google Scholar
• Donoho, D., and Jin, J. (2015), “Higher Criticism for Large-Scale Inference, Especially for Rare and Weak Effects,” Statistical Science, 30, 1–25.
   Web of Science ®Google Scholar
• Fisher, R. A. (1934), Statistical Methods for Research Workers (5th ed.), Edinburgh: Oliver and Boyd.
   Google Scholar
• Flenner, A., and Hewer, G. (2011), “A Helmholtz Principle Approach to Parameter-Free Change Detection and Coherent Motion using Exchangeable Random Variables,” SIAM Journal on Imaging Sciences, 4, 243–276. DOI: 10.1137/090772344.
   Web of Science ®Google Scholar
• Hall, P., and Jin, J. (2008), “Properties of Higher Criticism under Strong Dependence,” Annals of Statistics, 36, 381–402.
   Web of Science ®Google Scholar
• Hall, P., and Jin, J. (2010). “Innovated Higher Criticism for Detecting Sparse Signals in Correlated Noise,” Annals of Statistics, 38, 1686–1732.
   Web of Science ®Google Scholar
• Hettmansperger, T. P. (1984), Statistical Inference based on Ranks, Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics, New York: Wiley.
   Google Scholar
• Huang, L., Kulldorff, M., and Gregorio, D. (2007), “A Spatial Scan Statistic for Survival Data,” Biometrics, 63, 109–118. DOI: 10.1111/j.1541-0420.2006.00661.x.
   PubMed Web of Science ®Google Scholar
• Ingster, Y. I. (1997), “Some Problems of Hypothesis Testing Leading to Infinitely Divisible Distributions,” Mathematical Methods of Statistics, 6, 47–69.
   Google Scholar
• Kulldorff, M., Heffernan, R., Hartman, J., Assuncao, R., and Mostashari, F. (2005), “A Space-Time Permutation Scan Statistic for Disease Outbreak Detection,” PLOS Medicine, 2, 216–224. DOI: 10.1371/journal.pmed.0020059.
   Web of Science ®Google Scholar
• Kulldorff, M., Huang, L., and Konty, K. (2009), “A Scan Statistic for Continuous Data based on the Normal Probability Model,” International Journal of Health Geographics, 8, 1–9. DOI: 10.1186/1476-072X-8-58.
   PubMed Web of Science ®Google Scholar
• Kurt, M. N., Yilmaz, Y., and Wang, X. (2020), “Real-Time Nonparametric Anomaly Detection in High-Dimensional Settings,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 2463–2476. DOI: 10.1109/TPAMI.2020.2970410.
   Web of Science ®Google Scholar
• Lehmann, E. L., and Romano, J. P. (2005), Testing Statistical Hypotheses. Springer Texts in Statistics (3rd ed.), New York: Springer.
   Google Scholar
• Lexa, M. A., Rozell, C. J., Sinanovic, S., and Johnson, D. H. (2004), “To Cooperate or Not to Cooperate: Detection Strategies in Sensor Networks,” in 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’04) (Vol. 3), pp. 841–844. DOI: 10.1109/ICASSP.2004.1326676.
   Google Scholar
• Mei, Y. (2008), “Asymptotic Optimality Theory for Decentralized Sequential Hypothesis Testing in Sensor Networks,” IEEE Transactions on Information Theory, 54, 2072–2089. DOI: 10.1109/TIT.2008.920217.
   Web of Science ®Google Scholar
• Mikosch, T., and Nagaev, A. V. (1998), “Large Deviations of Heavy-Tailed Sums with Applications in Insurance,” Extremes, 1, 81–110.
   Google Scholar
• Nichols, T. E., and Holmes, A. P. (2002), “Nonparametric Permutation Tests for Functional Neuroimaging: A Primer with Examples,” Human Brain Mapping, 15, 1–25. DOI: 10.1002/hbm.1058.
   PubMed Web of Science ®Google Scholar
• Patwari, N., and Hero, A. O. (2003), “Hierarchical Censoring for Distributed Detection in Wireless Sensor Networks,” in 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. (ICASSP ’03) (Vol. 4), pp. 848–851.
   Google Scholar
• Romano, J., Shaikh, A., and Wolf, M. (2008), “Control of the False Discovery Rate under Dependence using the Bootstrap and Subsampling,” TEST, 17, 417–442. DOI: 10.1007/s11749-008-0126-6.
   Web of Science ®Google Scholar
• Romano, J. P., and Wolf, M. (2007), “Control of Generalized Error Rates in Multiple Testing,” Annals of Statistics, 35, 1378–1408.
   Web of Science ®Google Scholar
• Sabatti, C., Service, S. K., Hartikainen, A. L., Pouta, A., Ripatti, S., Brodsky, J., Jones, C. G., Zaitlen, N. A., Varilo, T., Kaakinen, M., Sovio, U., Ruokonen, A., Laitinen, J., Jakkula, E., Coin, L., Hoggart, C., Collins, A., Turunen, H., Gabriel, S., Elliot, P., McCarthy, M. I., Daly, M. J., Järvelin, M. R., Freimer, N. B., and Peltonen, L. (2009), “Genome-Wide Association Analysis of Metabolic Traits in a Birth Cohort from a Founder Population,” Nature Genetics, 41, 35–46. DOI: 10.1038/ng.271.
   PubMed Web of Science ®Google Scholar
• Thomopoulos, S. C. A., Viswanathan, R., and Bougoulias, D. K. (1989), “Optimal Distributed Decision Fusion,” IEEE Transactions on Aerospace and Electronic Systems, 25, 761–765. DOI: 10.1109/7.42092.
   Google Scholar
• Walther, G. (2010), “Optimal and Fast Detection of Spatial Clusters with Scan Statistics,” Annals of Statistics, 38, 1010–1033.
   Web of Science ®Google Scholar
• Wu, Z., Sun, Y., He, S., Cho, J., Zhao, H., and Jin, J. (2014), “Detection Boundary and Higher Criticism Approach for Rare and Weak Genetic Effects,” Annals of Applied Statistics, 8, 824–851.
   Web of Science ®Google Scholar
• Yu, L., Yuan, L., Qu, G., and Ephremides, A. (2006), “Energy-Driven Detection Scheme with Guaranteed Accuracy,” in 2006 5th International Conference on Information Processing in Sensor Networks, IPSN 2006, pp. 284–291.
   Google Scholar
• Zou, S., Liang, Y., Poor, H. V., and Shi, X. (2017), “Nonparametric Detection of Anomalous Data Streams,” IEEE Transactions on Signal Processing, 65, 5785–5797. DOI: 10.1109/TSP.2017.2733472.
   Web of Science ®Google Scholar