A high dimensional functional time series approach to evolution outlier detection for grouped smart meters

References

• Angelos, E. W. S., O. R. Saavedra, O. A. C. Cortés, and A. N. de Souza. 2011. Detection and identification of abnormalities in customer consumptions in power distribution systems. IEEE Transactions on Power Delivery 26 (4):2436–42. doi: 10.1109/TPWRD.2011.2161621.
   Web of Science ®Google Scholar
• Angiulli, F., and C. Pizzuti. 2002. Fast outlier detection in high dimensional spaces. In Principles of data mining and knowledge discovery, ed. T. Elomaa, H. Mannila, and H. Toivonen, 15–27. Berlin, Heidelberg: Springer Berlin Heidelberg.
   Google Scholar
• Arribas-Gil, A., and J. Romo. 2014. Shape outlier detection and visualization for functional data: The outliergram. Biostatistics (Oxford, England) 15 (4):603–19. doi: 10.1093/biostatistics/kxu006.
   PubMed Web of Science ®Google Scholar
• Blázquez-García, A., A. Conde, U. Mori, and J. A. Lozano. 2022. A review on outlier/anomaly detection in time series data. ACM Computing Surveys 54 (3):1–33. doi: 10.1145/3444690.
   Web of Science ®Google Scholar
• Breunig, M. M., H.-P. Kriegel, R. T. Ng, and J. Sander. 2000. LOF: identifying density-based local outliers. ACM SIGMOD Record 29 (2):93–104. doi: 10.1145/335191.335388.
   Web of Science ®Google Scholar
• Dryden, I. L., and K. V. Mardia. 2016. Statistical shape analysis, with applications in R. 2nd ed. Wiley Series in Probability and Statistics. Chichester: John Wiley and Sons.
   Google Scholar
• Elías, A., R. Jiménez, A. M. Paganoni, and L. M. Sangalli. 2022. Integrated depths for partially observed functional data. Journal of Computational and Graphical Statistics 1–12. doi: 10.1080/10618600.2022.2070171.
   Web of Science ®Google Scholar
• Ferraty, F., and P. Vieu. 2006. Nonparametric functional data analysis: Theory and practice. New York: Springer-Verlag.
   Google Scholar
• Fraiman, R., and G. Muniz. 2001. Trimmed means for functional data. Test 10 (2):419–40. doi: 10.1007/BF02595706.
   Web of Science ®Google Scholar
• Gao, Y., H. L. Shang, and Y. Yang. 2017. High-dimensional functional time series forecasting. In Functional statistics and related fields, ed. G. Aneiros, E. G. Bongiorno, R. Cao, and P. Vieu, 131–6. Cham: Springer International Publishing.
   Google Scholar
• Gao, Y., H. L. Shang, and Y. Yang. 2019. High-dimensional functional time series forecasting: An application to age-specific mortality rates. Journal of Multivariate Analysis 170:232–43 (Special Issue on Functional Data Analysis and Related Topics). doi: 10.1016/j.jmva.2018.10.003.
   Web of Science ®Google Scholar
• Gaur, M., S. Makonin, I. V. Bajić, and A. Majumdar. 2019. Performance evaluation of techniques for identifying abnormal energy consumption in buildings. IEEE Access 7:62721–33. doi: 10.1109/ACCESS.2019.2915641.
   Web of Science ®Google Scholar
• Gijbels, I., and S. Nagy. 2017. On a general definition of depth for functional data. Statistical Science 32 (4):630–9. doi: 10.1214/17-STS625.
   Web of Science ®Google Scholar
• Harris, T., J. D. Tucker, B. Li, and L. Shand. 2021. Elastic depths for detecting shape anomalies in functional data. Technometrics 63 (4):466–76. doi: 10.1080/00401706.2020.1811156.
   Web of Science ®Google Scholar
• Himeur, Y., K. Ghanem, A. Alsalemi, F. Bensaali, and A. Amira. 2021. Artificial intelligence based anomaly detection of energy consumption in buildings: A review, current trends and new perspectives.” Applied Energy 287:116601.
   Google Scholar
• Hörmann, S., and P. P. Kokoszka. 2012. Functional time series: Handbook of statistics, Vol. 30, 157–86. Netherlands: Elsevier B.V.
   Google Scholar
• Hubert, M., and E. Vandervieren. 2008. An adjusted boxplot for skewed distributions. Computational Statistics & Data Analysis 52 (12):5186–201. doi: 10.1016/j.csda.2007.11.008.
   Web of Science ®Google Scholar
• Hyndman, R. J., and H. L. Shang. 2010. Rainbow plots, bagplots, and boxplots for functional data. Journal of Computational and Graphical Statistics 19 (1):29–45. doi: 10.1198/jcgs.2009.08158.
   Web of Science ®Google Scholar
• Hyndman, R. J., E. Wang, and N. Laptev. 2015. Large-scale unusual time series detection. In 2015 IEEE International Conference on Data Mining Workshop (ICDMW), Atlantic City, NJ, USA, November 14, pp. 1616–9.
   Google Scholar
• Jin, W., A. K. H. Tung, J. Han, and W. Wang. 2006. Ranking outliers using symmetric neighborhood relationship. In Advances in knowledge discovery and data mining, ed. W.-K. Ng, M. Kitsuregawa, J. Li, and K. Chang, 577–93. Berlin, Heidelberg: Springer Berlin Heidelberg.
   Google Scholar
• Jindal, A., A. Dua, K. Kaur, M. Singh, N. Kumar, and S. Mishra. 2016. Decision tree and SVM-based data analytics for theft detection in smart grid. IEEE Transactions on Industrial Informatics 12 (3):1005–16. doi: 10.1109/TII.2016.2543145.
   Web of Science ®Google Scholar
• Kanal, A. 2020. Kaggle: Solar power generation data. https://www.kaggle.com/anikannal/solar-power-generation-data/metadata (accessed March 26, 2021).
   Google Scholar
• Liu, S., Y. Liang, J. Wang, T. Jiang, W. Sun, and Y. Rui. 2020. Identification of stealing electricity based on big data analysis. 2020 The 7th International Conference on Power and Energy Systems Engineering. Energy Reports 6:731–8. doi: 10.1016/j.egyr.2020.11.138.
   Google Scholar
• López-Pintado, S., and J. Romo. 2009. On the concept of depth for functional data. Journal of the American Statistical Association 104 (486):718–34. doi: 10.1198/jasa.2009.0108.
   Web of Science ®Google Scholar
• López-Pintado, S., and J. Romo. 2011. A half-region depth for functional data. Computational Statistics & Data Analysis 55 (4):1679–95. doi: 10.1016/j.csda.2010.10.024.
   Web of Science ®Google Scholar
• Marron, J. S., J. O. Ramsay, L. M. Sangalli, and A. Srivastava. 2015. Functional data analysis of amplitude and phase variation. Statistical Science 30 (4):468–84. http://www.jstor.org/stable/24780816.
   Web of Science ®Google Scholar
• Marron, J. S., and A. B. Tsybakov. 1995. Visual error criteria for qualitative smoothing. Journal of the American Statistical Association 90 (430):499–507. doi: 10.1080/01621459.1995.10476541.
   Web of Science ®Google Scholar
• Meeker, W. Q., and Y. Hong. 2014. Reliability meets big data: Opportunities and challenges. Quality Engineering 26 (1):102–16. doi: 10.1080/08982112.2014.846119.
   Web of Science ®Google Scholar
• Mosler, K. 2013. Depth statistics. In Robustness and complex data structures, ed. G. Biau, 17–34. Heidelberg: Springer.
   Google Scholar
• Nagy, S., I. Gijbels, M. Omelka, and D. Hlubinka. 2016. Integrated depth for functional data: Statistical properties and consistency. ESAIM. Probability and Statistics 20:95–130.
   Web of Science ®Google Scholar
• Narisetty, N. N., and V. N. Nair. 2016. Extremal depth for functional data and applications. Journal of the American Statistical Association 111 (516):1705–14. doi: 10.1080/01621459.2015.1110033.
   Web of Science ®Google Scholar
• Ramaswamy, S., R. Rastogi, and K. Shim. 2000. Efficient algorithms for mining outliers from large data sets. ACM SIGMOD Record 29 (2):427–38. doi: 10.1145/335191.335437.
   Web of Science ®Google Scholar
• Ramsay, J. O., and B. W. Silverman. 2005. Functional data analysis. Springer Series in Statistics. Berlin: Springer.
   Google Scholar
• Rasmussen, C. E., and C. K. I. Williams. 2005. Gaussian processes for machine learning. Cambridge, MA: MIT Press. doi: 10.7551/mitpress/3206.001.0001.
   Google Scholar
• Raña, P., G. Aneiros, and J. M. Vilar. 2015. Detection of outliers in functional time series. Environmetrics 26 (3):178–91. doi: 10.1002/env.2327.
   Web of Science ®Google Scholar
• Sangalli, L. M. 2018. The role of statistics in the era of big data. Statistics & Probability Letters 136:1–3. https://www.sciencedirect.com/science/article/pii/S016771521830155X.
   Web of Science ®Google Scholar
• Sangalli, L. M. 2020. A novel approach to the analysis of spatial and functional data over complex domains. Quality Engineering 32 (2):181–90. doi: 10.1080/08982112.2019.1659357.
   Web of Science ®Google Scholar
• Schölkopf, B., R. Williamson, A. Smola, J. Shawe-Taylor, and J. Platt. 1999. Support vector method for novelty detection. In Proceedings of the 12th International Conference on Neural Information Processing Systems, NIPS’99, 582–8. Cambridge, MA, USA: MIT Press.
   Google Scholar
• Srivastava, A., and E. P. Klassen. 2016. Functional and shape data analysis. New York, NY: Springer.
   Google Scholar
• Street, P. 2012. Real energy. Real constumers. In real time. http://www.pecanstreet.org/energy/.
   Google Scholar
• Sun, L., K. Zhou, X. Zhang, and S. Yang. 2018. Outlier data treatment methods toward smart grid applications. IEEE Access 6:39849–59. doi: 10.1109/ACCESS.2018.2852759.
   Web of Science ®Google Scholar
• Sun, W., and J. Hou. 2016. A MPRM-based approach for fault diagnosis against outliers. Neurocomputing 190:147–54. doi: 10.1016/j.neucom.2016.01.023.
   Web of Science ®Google Scholar
• Sun, Y., and M. G. Genton. 2011. Functional boxplots. Journal of Computational and Graphical Statistics 20 (2):316–34. doi: 10.1198/jcgs.2011.09224.
   Web of Science ®Google Scholar
• Sun, Y., M. G. Genton, and D. C. Nychka. 2012. Exact fast computation of band depth for large functional datasets: How quickly can one million curves be ranked? Stat 1 (1):68–74. doi: 10.1002/sta4.8.
   Google Scholar
• Tanasa, D., and B. Trousse. 2004. Advanced data preprocessing for intersites Web usage mining. IEEE Intelligent Systems 19 (2):59–65. doi: 10.1109/MIS.2004.1274912.
   Web of Science ®Google Scholar
• Tang, J., Z. Chen, A. Wai-Chee Fu, and D. W. Cheung. 2002. Enhancing effectiveness of outlier detections for low density patterns. In Advances in knowledge discovery and data mining, ed. M.-S. Chen, P. S. Yu, and B. Liu, 535–48. Berlin, Heidelberg: Springer Berlin Heidelberg.
   Google Scholar
• Tukey, J. W. 1975. Mathematics and the picturing of data. In Proceedings of the International Congress of Mathematics, Vancouver, 1974, Vol. 2, 523–31.
   Google Scholar
• Vallakati, R., A. Mukherjee, and P. Ranganathan. 2015. A density based clustering scheme for situational awareness in a smart-grid. In 2015 IEEE International Conference on Electro/Information Technology (EIT), Dekalb, IL, USA, pp. 346–50.
   Google Scholar
• Wang, Y., Q. Chen, T. Hong, and C. Kang. 2019a. Review of smart meter data analytics: Applications, methodologies, and challenges. IEEE Transactions on Smart Grid 10 (3):3125–48. doi: 10.1109/TSG.2018.2818167.
   Web of Science ®Google Scholar
• Wang, Z., G. Li, X. Wang, C. Chen, and H. Long. 2019b. Analysis of 10kV non-technical loss detection with data-driven approaches. In 2019 IEEE Innovative Smart Grid Technologies – Asia (ISGT Asia), Chengdu, China, pp. 4154–8. doi: 10.1109/ISGT-Asia.2019.8881733.
   Google Scholar
• Xie, W., S. Kurtek, K. Bharath, and Y. Sun. 2017. A geometric approach to visualization of variability in functional data. Journal of the American Statistical Association 112 (519):979–93. doi: 10.1080/01621459.2016.1256813.
   Web of Science ®Google Scholar
• Yin, T., S. S. Wulff, J. W. Pierre, and T. J. Robinson. 2019. A case study on the use of data mining for detecting and classifying abnormal power system modal behaviors. Quality Engineering 31 (2):314–33. doi: 10.1080/08982112.2018.1530356.
   Web of Science ®Google Scholar
• Zuo, Y., and R. Serfling. 2000. General notions of statistical depth function. The Annals of Statistics 28 (2):461–82. http://www.jstor.org/stable/2674037.
   Web of Science ®Google Scholar