Interpretable SAM-kNN Regressor for Incremental Learning on High-Dimensional Data Streams

References

• Andrade, T., B. Cancela, and J. Gama. 2020. Discovering locations and habits from human mobility data. Annals of Telecommunications 75 (9):505–1105. doi:10.1007/s12243-020-00807-x.
   Web of Science ®Google Scholar
• Barddal, J. P., H. Murilo Gomes, F. Enembreck, and B. Pfahringer. 2017. A survey on feature drift adaptation: Definition, benchmark, challenges and future directions. The Journal of Systems and Software 127:278–94. doi:10.1016/j.jss.2016.07.005.
   Web of Science ®Google Scholar
• Bellet, A., A. Habrard, and M. Sebban. 2013. “A survey on metric learning for feature vectors and structured data.” https://arxiv.org/abs/1306.6709.
   Google Scholar
• Bouzid, S., and M. Ramdani. 2013. “Sensor fault detection and diagnosis in drinking water distribution networks.” In 2013 8th International Workshop on Systems, Signal Processing and their Applications (WoSSPA), 378–83. Algiers, Algeria.
   Google Scholar
• Castellani, A., S. Schmitt, and B. Hammer. 2021. Estimating the electrical power output of industrial devices with end-to-end time-series classification in the presence of label noise. In Machine learning and knowledge discovery in databases. research track, ed. N. Oliver, F. Pérez-Cruz, S. Kramer, J. Read, and J. A. Lozano, 469–84. Basel, Switzerland: Springer International Publishing.
   Google Scholar
• Domingos, P., and G. Hulten. 2000. “Mining high-speed data streams.” In Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, 71–80. Boston, Massachusetts, United States.
   Google Scholar
• Eliades, D. G., and M. M. Polycarpou. 2012. Leakage fault detection in district metered areas of water distribution systems. Journal of Hydroinformatics 14 (4):992–1005. doi:10.2166/hydro.2012.109.
   Web of Science ®Google Scholar
• El-Zahab, S., and T. Zayed. 2019. Leak detection in water distribution networks: An introductory overview. Smart Water 4. doi:10.1186/s40713-019-0017-x.
   Google Scholar
• Göpfert, J. P., B. Hammer, and H. Wersing. 2018. “Mitigating concept drift via rejection.” In International Conference on Artificial Neural Networks, 456–67. Rhodes, Greece: Springer.
   Google Scholar
• Grbovic, M., and S. Vucetic. 2009. “Learning vector quantization with adaptive prototype addition and removal.” In 2009 international joint conference on neural networks, 994–1001. Atlanta, Georgia, United States: IEEE.
   Google Scholar
• Hammer, B., and T. Villmann. 2002. Generalized relevance learning vector quantization. Neural Networks: The Official Journal of the International Neural Network Society 15:1059–68. doi:10.1016/S0893-6080(02)00079-5.
   PubMed Web of Science ®Google Scholar
• Hoi, S. C., D. Sahoo, L. Jing, and P. Zhao. 2021. Online learning: A comprehensive survey. Neurocomputing 459:249–89. doi:10.1016/j.neucom.2021.04.112.
   Web of Science ®Google Scholar
• Jakob, J., A. Artelt, M. Hasenjäger, and B. Hammer. 2022. “SAM-kNN regressor for online learning in water distribution networks.” In International Conference on Artificial Neural Networks, 752–62. Bristol, United Kingdom. 10.1007/978-3-031-15934-3_62.
   Google Scholar
• Jakob, J., M. Hasenjäger, and B. Hammer. 2021. “On the suitability of incremental learning for regression tasks in exoskeleton control.” In IEEE Symposium Series on Computational Intelligence, SSCI 2021, Orlando, FL, USA, December 5-7, 2021, 1–8. IEEE. 10.1109/SSCI50451.2021.9660138.
   Google Scholar
• Klise, K. A., R. Murray, and T. Haxton. 2018. An Overview of the Water Network Tool for Resilience (WNTR). Computing and Control in the Water Industry, Ontario, Kingston, Canada.
   Google Scholar
• Kulis, B. 2012. Metric Learning: A Survey. Foundations and Trends in Machine Learning 5:287–364. doi:10.1561/2200000019.
   Google Scholar
• Kummert, J., A. Schulz, and B. Hammer. 2023. Metric learning with self adjusting memory for explaining feature drift. Submitted to SN Computer Science.
   Google Scholar
• Losing, V., B. Hammer, and H. Wersing. 2015. “Interactive online learning for obstacle classification on a mobile robot.” In 2015 international joint conference on neural networks (ijcnn), 1–8. Killarney, Ireland: IEEE.
   Google Scholar
• Losing, V., B. Hammer, and H. Wersing. 2016. “KNN classifier with self adjusting memory for heterogeneous concept drift.” In 2016 IEEE 16th international conference on data mining (ICDM). 291–300. Barcelona, Spain: IEEE.
   Google Scholar
• Losing, V., B. Hammer, and H. Wersing. 2018a. Incremental on-line learning: A review and comparison of state of the art algorithms. Neurocomputing 275:1261–74. doi:10.1016/j.neucom.2017.06.084.
   Web of Science ®Google Scholar
• Losing, V., B. Hammer, and H. Wersing. 2018b. Tackling heterogeneous concept drift with the self-adjusting memory (SAM). Knowledge and Information Systems 54 (1):171–201. doi:10.1007/s10115-017-1137-y.
   Web of Science ®Google Scholar
• Losing, V., B. Hammer, H. Wersing, and A. Bifet. 2020. “Randomizing the self-adjusting memory for enhanced handling of concept drift.” In 2020 International Joint Conference on Neural Networks (IJCNN), 1–8. Glasgow, United Kingdom: IEEE.
   Google Scholar
• Martinetz, T., K. Schulten, et al. 1991. A” neural-gas“network learns topologies Artificial Neural Networks. 1, 397–402 .
   Google Scholar
• Oladeji, I., R. Zamora, and T. Tjing Lie. 2021. An online security prediction and control framework for modern power grids. Energies 14 (20):6639. doi:10.3390/en14206639.
   Web of Science ®Google Scholar
• Pau, D., A. Khiari, and D. Denaro. 2021. “Online learning on tiny micro-controllers for anomaly detection in water distribution systems.” In 2021 IEEE 11th International Conference on Consumer Electronics (ICCE-Berlin), 1–6. Berlin, Germany: IEEE.
   Google Scholar
• Reppa, V., M. M. Polycarpou, and C. G. Panayiotou. 2013. “Multiple sensor fault detection and isolation for large-scale interconnected nonlinear systems.” In 2013 European Control Conference (ECC). 1952–57. Zurich, Switzerland.
   Google Scholar
• Reppa, V., M. M. Polycarpou, and C. G. Panayiotou. 2016. Sensor Fault Diagnosis. Foundations and Trends® in Systems and Control 3 (1–2):1–248. doi:10.1561/2600000007.
   Google Scholar
• Roseberry, M., B. Krawczyk, and A. Cano. 2019. Multi-label punitive kNN with self-adjusting memory for drifting data streams. ACM Transactions on Knowledge Discovery from Data (TKDD) 13 (6):1–31. doi:10.1145/3363573.
   Web of Science ®Google Scholar
• Rosenblatt, F. 1958. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review 65 (6):386. doi:10.1037/h0042519.
   PubMed Web of Science ®Google Scholar
• Santos-Ruiz, I., F.R. López-Estrada, V. Puig, and J. Blesa. 2019. “Estimation of node pressures in water distribution networks by Gaussian process regression.” In 2019 4th Conference on Control and Fault Tolerant Systems (SysTol) Casablanca, Morocco, 50–55. IEEE.
   Google Scholar
• Schneider, P., M. Biehl, and B. Hammer. 2009. Adaptive relevance matrices in learning vector quantization. Neural Computation 21 (12). doi:10.1162/neco.2009.11-08-908.
   Web of Science ®Google Scholar
• Song, L., C. Tekin, and M. Van Der Schaar. 2014. Online learning in large-scale contextual recommender systems. IEEE Transactions on Services Computing 9 (3):433–45. doi:10.1109/TSC.2014.2365795.
   Web of Science ®Google Scholar
• Vaquet, V., A. Artelt, J. Brinkrolf, and B. Hammer. 2022. “Taking care of our drinking water: dealing with sensor faults in water distribution networks.” In International Conference on Artificial Neural Networks, 682–93. Bristol, United Kingdom: Springer.
   Google Scholar
• Vaquet, V., and B. Hammer. 2020. “Balanced sam-knn: Online learning with heterogeneous drift and imbalanced data.” In International Conference on Artificial Neural Networks, 850–62. Bratislava, Slovakia: Springer.
   Google Scholar
• Vijayakumar, S., A. D’souza, and S. Schaal. 2005. Incremental online learning in high dimensions. Neural Computation 17 (12):2602–34. doi:10.1162/089976605774320557.
   PubMed Web of Science ®Google Scholar
• Vrachimis, S. G., D. G. Eliades, R. Taormina, A. Ostfeld, Z. Kapelan, S. Liu, M. Kyriakou, P. Pavlou, M. Qiu, and M. M. Polycarpou. 2020. BattLeDIM: Battle of the leakage detection and isolation methods Journal of Water Resources Planning and Managment. 148: 421–438 .
   Web of Science ®Google Scholar
• Weinberger, K., and L. Saul. 2009. Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research 10:207–44.
   Web of Science ®Google Scholar
• Weinberger, K., and G. Tesauro. 2007. “Metric learning for kernel regression Journal of Machine Learning Research. 2: 612–19.
   Google Scholar
• Yamaguchi, A., S. Maya, T. Inagi, and K. Ueno. 2018. “OPOSSAM: Online prediction of stream data using self-adaptive memory.” In 2018 IEEE International Conference on Big Data (Big Data), 2355–64. Seattle, Washington, United States.
   Google Scholar
• Yang, Y., J. Che, L. YanYing, Y. Zhao, and S. Zhu. 2016. An incremental electric load forecasting model based on support vector regression. Energy 113:796–808. https://www.sciencedirect.com/science/article/pii/S0360544216310167.
   Web of Science ®Google Scholar
• Yan, J., C. Tian, Y. Wang, and J. Huang. 2012. “Online incremental regression for electricity price prediction.” In Proceedings of 2012 IEEE International Conference on Service Operations and Logistics, and Informatics, SOLI 2012, Suzhou, China, July 8-10, 2012, 31–35. IEEE. 10.1109/SOLI.2012.6273500.
   Google Scholar
• Yipeng, W., and S. Liu. 2017. A review of data-driven approaches for burst detection in water distribution systems. Urban Water Journal 14 (9):972–83. doi:10.1080/1573062X.2017.1279191.
   Web of Science ®Google Scholar
• Zhao, D., and Y. Sing Koh. 2020. Feature drift detection in evolving data streams. In Database and expert systems applications, ed. S. Hartmann, J. Küng, G. Kotsis, A. M. Tjoa, and C. Ismail Khalil, 335–49. Basel, Switzerland: Springer International Publishing.
   Google Scholar
• Zliobaite, I., M. Pechenizkiy, and J. Gama. 2016 An Overview of Concept Drift Applications Kacprzyk, Janusz. Studies in Big Data, 16, 91–114. Basel, Switzerland: Springer International Publishing.
   Google Scholar