An intelligent unsupervised anomaly detection in videos using inception capsule auto encoder

References

• Zhu S, Chen C, Sultani W. Video anomaly detection for smart surveillance. In: Mubarak Shah, editor. Computer vision: a reference guide. Cham: Springer International Publishing; 2020. p. 1–8.
   Google Scholar
• Verma KK, Singh BM, Dixit A. A review of supervised and unsupervised machine learning techniques for suspicious behavior recognition in intelligent surveillance system. Int J Inf Technol. 2019: 1–14.
   Google Scholar
• Huang C, Wu Z, Wen J, et al. Abnormal event detection using deep contrastive learning for intelligent video surveillance system. IEEE Trans Ind Inf. 2022;18(8):5171–5179.
   Web of Science ®Google Scholar
• Hao Y, Li J, Wang N, et al. Spatiotemporal consistency-enhanced network for video anomaly detection. Pattern Recognit. 2022;121:108232.
   Web of Science ®Google Scholar
• Doshi K, Yilmaz Y. Fast unsupervised anomaly detection in traffic videos. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops; 2020: 624–625.
   Google Scholar
• Kavitha M, Srinivas PVVS, Kalyampudi PSL, et al. Machine learning techniques for anomaly detection in smart healthcare. 2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA); 2021: 1350–1356; IEEE.
   Google Scholar
• Nayak R, Pati UC, Das SK. A comprehensive review on deep learning-based methods for video anomaly detection. Image Vis Comput. 2021;106:104078.
   Web of Science ®Google Scholar
• Zhang Q, Feng G, Wu H. Surveillance video anomaly detection via non-local u-net frame prediction. Multimed Tools Appl. 2022;81(19):1–16.
   PubMed Web of Science ®Google Scholar
• Mansour RF, Escorcia-Gutierrez J, Gamarra M, et al. Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model. Image Vis Comput. 2021;112:104229.
   Web of Science ®Google Scholar
• Himeur Y, Ghanem K, Alsalemi A, et al. Artificial intelligence based anomaly detection of energy consumption in buildings: a review, current trends and new perspectives. Appl Energy. 2021;287:116601.
   Web of Science ®Google Scholar
• Ren J, Xia F, Liu Y, et al. Deep video anomaly detection: opportunities and challenges. 2021 International Conference on Data Mining Workshops (ICDMW); 2021: 959–966; IEEE.
   Google Scholar
• Chriki A, Touati H, Snoussi H, et al. Deep learning and handcrafted features for one-class anomaly detection in UAV video. Multimed Tools Appl. 2021;80(2):2599–2620.
   Web of Science ®Google Scholar
• Rezaee K, Rezakhani SM, Khosravi MR, et al. A survey on deep learning-based real-time crowd anomaly detection for secure distributed video surveillance. Pers Ubiquitous Comput. 2021: 1–17.
   Google Scholar
• Li X, Cai Z. Anomaly detection techniques in surveillance videos. 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI); 2016: 54–59; IEEE.
   Google Scholar
• Ahmed M, Gupta A, Goel M, et al. Optimized convolutional neural network model for fire detection in surveillance videos. 2022 6th International Conference on Intelligent Computing and Control Systems (ICICCS); 2022: 1121–1125; IEEE.
   Google Scholar
• Audibert J, Michiardi P, Guyard F, et al. Usad: unsupervised anomaly detection on multivariate time series. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; 2020. August: 3395–3404.
   Google Scholar
• Chang Y, Tu Z, Xie W, et al. Video anomaly detection with spatio-temporal dissociation. Pattern Recognit. 2022;122:108213.
   Web of Science ®Google Scholar
• Schlegl T, Seeböck P, Waldstein SM, et al. f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks. Med Image Anal. 2019;54:30–44.
   PubMed Web of Science ®Google Scholar
• Chang Y, Tu Z, Xie W, et al. Clustering driven deep autoencoder for video anomaly detection. Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16; 2020; 329-345; Springer International Publishing.
   Google Scholar
• Kosiorek A, Sabour S, Teh YW, et al. Stacked capsule autoencoders. Adv Neural Inf Process Syst. 2019: 32.
   Google Scholar
• Elhalwagy A, Kalganova T. Multi-channel LSTM-capsule autoencoder network for anomaly detection on multivariate data. Appl Sci. 2022;12(22):11393.
   Google Scholar
• Kiran BR, Thomas DM, Parakkal R. An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos. J Imaging. 2018;4(2):36.
   Web of Science ®Google Scholar
• Murugesan M, Thilagamani S. Efficient anomaly detection in surveillance videos based on multi layer perception recurrent neural network. Microprocess Microsys. 2020;79:103303.
   Web of Science ®Google Scholar
• Aberkane S, Elarbi M. Deep reinforcement learning for real-world anomaly detection in surveillance videos. 2019 6th International Conference on Image and Signal Processing and Their Applications (ISPA); 2019; IEEE.
   Google Scholar
• Roka S, Diwakar M, Karanwal S. A review in anomalies detection using deep learning. Proceedings of Third International Conference on Sustainable Computing; 2022; 329-338; Springer, Singapore.
   Google Scholar
• Khaleghi A, Moin MS. Improved anomaly detection in surveillance videos based on a deep learning method. In 2018 8th Conference of AI & Robotics and 10th RoboCup Iranopen International Symposium (IRANOPEN), IEEE. 2018: 73–81.
   Google Scholar
• Ul Amin S, Ullah M, Sajjad M, et al. Eadn: an efficient deep learning model for anomaly detection in videos. Mathematics. 2022;10(9):1555.
   Web of Science ®Google Scholar
• Chandrakala S, Srinivas V, Deepak K. Residual spatiotemporal autoencoder with skip connected and memory guided network for detecting video anomalies. Neural Process Lett. 2021;53(6):4677–4692.
   Web of Science ®Google Scholar
• Nawaratne R, Alahakoon D, Silva DD, et al. Spatiotemporal anomaly detection using deep learning for real-time video surveillance. IEEE Trans Ind Inf. 2020;16(1):393–402.
   Web of Science ®Google Scholar
• Khaire P, Kumar P. A semi-supervised deep learning based video anomaly detection framework using RGB-D for surveillance of real-world critical environments. Forensic Sci Int: Digital Invest. 2022;40:301346.
   Web of Science ®Google Scholar
• Sarker MI, Losada-Gutiérrez C, Marrón-Romera M, et al. Semi-supervised anomaly detection in video-surveillance scenes in the wild. Sensors. 2021;21(12):3993.
   PubMed Web of Science ®Google Scholar
• Ullah W, Ullah A, Hussain T, et al. Artificial intelligence of things-assisted two-stream neural network for anomaly detection in surveillance big video data. Future Gener Comput Syst. 2022;129:286–297.
   Web of Science ®Google Scholar
• Ullah W, Hussain T, Khan ZA, et al. Intelligent dual stream CNN and echo state network for anomaly detection. Knowl Based Syst. 2022;253:109456.
   Web of Science ®Google Scholar
• Mangai P, Geetha MK, Kumaravelan G. Temporal features-based anomaly detection from surveillance videos using deep learning techniques. 2022 s International Conference on Artificial Intelligence and Smart Energy (ICAIS); 2022: 490–497; IEEE.
   Google Scholar
• Chen D, Yue L, Chang X, et al. NM-GAN: noise-modulated generative adversarial network for video anomaly detection. Pattern Recognit. 2021;116:107969.
   Web of Science ®Google Scholar
• Lu B, Xu D, Huang B. Deep-learning-based anomaly detection for lace defect inspection employing videos in production line. Adv Eng Inf. 2022;51:101471.
   Web of Science ®Google Scholar
• Murugan BS, Elhoseny M, Shankar K, et al. Region-based scalable smart system for anomaly detection in pedestrian walkways. Comput Electr Eng. 2019;75:146–160.
   Web of Science ®Google Scholar
• Ratre A. Taylor series based compressive approach and firefly support vector neural network for tracking and anomaly detection in crowded videos. J Eng Res. 2019;7(4):115–137.
   Google Scholar
• Baradaran M, Bergevin R. A critical study on the recent deep learning based semi-supervised video anomaly detection methods. arXiv preprint arXiv:2111.01604. 2021: 1–28.
   Google Scholar
• Zhu C, Xu J, Feng D, et al. Edge-based video compression texture synthesis using generative adversarial network. IEEE Trans Circuits Syst Video Technol. 2022;32(10):7061–7076.
   Web of Science ®Google Scholar
• Mallick AK, Mukhopadhyay S. Video retrieval based on motion vector key frame extraction and spatial pyramid matching. 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN); 2019: 687–692; IEEE.
   Google Scholar
• Ilyas Z, Aziz Z, Qasim T, et al. A hybrid deep network based approach for crowd anomaly detection. Multimed Tools Appl. 2021;80(16):24053–24067.
   Web of Science ®Google Scholar
• Sabour S, Frosst N, Hinton GE. Dynamic routing between capsules. Adv Neural Inf Process Syst. 2017: 30.
   Google Scholar
• Dombetzki LA. An overview over capsule networks. Network Archit Serv. 2018: 1–11.
   Google Scholar
• Shen F, Zhou J, Huang Z, et al. Going deeper into OSNR estimation with CNN. In: Photonics (Vol. 8, No. 9, p. 402). MDPI.
   Google Scholar
• Zhu Z, Peng G, Chen Y, et al. A convolutional neural network based on a capsule network with strong generalization for bearing fault diagnosis. Neurocomputing. 2019;323:62–75.
   Web of Science ®Google Scholar
• Ma Z, Yun CB, Wan HP, et al. Probabilistic principal component analysis-based anomaly detection for structures with missing data. Struct Control Health Monit. 2021;28(5):e2698.
   Web of Science ®Google Scholar
• Finke T, Krämer M, Morandini A, et al. Autoencoders for unsupervised anomaly detection in high energy physics. J High Energy Phys. 2021;2021(6):1–32.
   PubMedGoogle Scholar
• Deepak K, Chandrakala S, Mohan CK. Residual spatiotemporal autoencoder for unsupervised video anomaly detection. Signal Image Video Process. 2021;15(1):215–222.
   Web of Science ®Google Scholar
• Pierezan J, Coelho LDS. Coyote optimization algorithm: a new metaheuristic for global optimization problems. 2018 IEEE Congress on Evolutionary Computation (CEC); 2018: 1–8; IEEE.
   Google Scholar
• Conner MM, Ebinger MR, Knowlton FF. Evaluating coyote management strategies using a spatially explicit, individual-based, socially structured population model. Ecol Modell. 2008;219(1-2):234–247.
   Web of Science ®Google Scholar
• CUHK-Avenue http://www.cse.cuhk.edu.hk/leojia/projects/detectabnormal/dataset.html.
   Google Scholar
• UCSD-Ped2 http://www.svcl.ucsd.edu/projects/anomaly/dataset.html.
   Google Scholar
• Live Videos (LV) dataset https://cvrleyva.wordpress.com/.
   Google Scholar
• Tang Y, Zhao L, Zhang S, et al. Integrating prediction and reconstruction for anomaly detection. Pattern Recognit Lett. 2020;129:123–130.
   Web of Science ®Google Scholar
• Qiang Y, Fei S, Jiao Y. Anomaly detection based on latent feature training in surveillance scenarios. IEEE Access. 2021;9:68108–68117.
   Web of Science ®Google Scholar
• Madan N, Farkhondeh A, Nasrollahi K, et al. Temporal cues from socially unacceptable trajectories for anomaly detection. Proceedings of the IEEE/CVF International Conference on Computer Vision; 2021; 2150–2158; Montreal, BC, Canada.
   Google Scholar
• Ganokratanaa T, Aramvith S, Sebe N. Unsupervised anomaly detection and localization based on deep spatiotemporal translation network. IEEE Access. 2020;8:50312–50329.
   Web of Science ®Google Scholar
• Singh K, Rajora S, Vishwakarma DK, et al. Crowd anomaly detection using aggregation of ensembles of fine-tuned convnets. Neurocomputing. 2020;371:188–198.
   Web of Science ®Google Scholar
• Wu C, Shao S, Tunc C, et al. An explainable and efficient deep learning framework for video anomaly detection. Clust Comput. 2021;25:1–23.
   Web of Science ®Google Scholar
• Zaheer MZ, Mahmood A, Shin H, et al. A self-reasoning framework for anomaly detection using video-level labels. IEEE Signal Process Lett. 2020;27:1705–1709.
   Web of Science ®Google Scholar
• Tian Y, Pang G, Chen Y, et al. Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. Proceedings of the IEEE/CVF International Conference on Computer Vision; 2021; 4975–4986; Montreal, BC, Canada.
   Google Scholar
• Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014: 1–14.
   Google Scholar
• Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016.
   Google Scholar
• Ullah W, Ullah A, Haq IU, et al. CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks. Multimed Tools Appl. 2021;80(11):16979–16995.
   Web of Science ®Google Scholar
• Zhong JX, Li N, Kong W, et al. Graph convolutional label noise cleaner: train a plug-and-play action classifier for anomaly detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2019: 1237–1246.
   Google Scholar
• Nor AKM, Pedapati SR, Muhammad M, et al. Abnormality detection and failure prediction using explainable Bayesian deep learning: methodology and case study with industrial data. Mathematics. 2022;10(4):554.
   Web of Science ®Google Scholar
• Li Y, Liu Y, Yu R, et al. Dual attention based spatial-temporal inference network for volleyball group activity recognition. Multimed Tools Appl. 2022;82(10):1–19.
   Web of Science ®Google Scholar
• Selicato L, Esposito F, Gargano G, et al. A new ensemble method for detecting anomalies in gene expression matrices. Mathematics. 2021;9(8):882.
   Web of Science ®Google Scholar
• Zahid Y, Tahir MA, Durrani MN. Ensemble learning using bagging and inception-V3 for anomaly detection in surveillance videos. 2020 IEEE International Conference on Image Processing (ICIP); 2020: 588–592; IEEE.
   Google Scholar
• Zhao X, Imandoust A, Khanzadeh M, et al. Automated anomaly detection of laser-based additive manufacturing using melt pool sparse representation and unsupervised learning. 2021 International Solid Freeform Fabrication Symposium; 2021; University of Texas at Austin.
   Google Scholar
• Liu W, Luo W, Lian D, et al. Future frame prediction for anomaly detection–a new baseline. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018; 6536-6545.
   Google Scholar
• Tian Y, Pang G, Chen Y, et al. (2021). Weakly-supervised video anomaly detection with contrastive learning of long and short-range temporal features.
   Google Scholar
• Gong D, Liu L, Le V, et al. Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. Proceedings of the IEEE/CVF International Conference on Computer Vision; 2019: 1705–1714.
   Google Scholar
• Yu G, Wang S, Cai Z, et al. Cloze test helps: effective video anomaly detection via learning to complete video events. Proceedings of the 28th ACM International Conference on Multimedia; 2020; 583-591.
   Google Scholar
• Park H, Noh J, Ham B. Learning memory-guided normality for anomaly detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2020: 14372–14381.
   Google Scholar
• Munyua JG, Wambugu GM, St N. (2021). A Survey of Deep Learning Solutions for Anomaly Detection in Surveillance Videos.
   Google Scholar
• Luo W, Liu W, Gao S. Normal graph: spatial temporal graph convolutional networks based prediction network for skeleton based video anomaly detection. Neurocomputing. 2021;444:332–337.
   Web of Science ®Google Scholar
• Ganokratanaa T, Aramvith S, Sebe N. Video anomaly detection using deep residual-spatiotemporal translation network. Pattern Recognit Lett. 2022;155:143–150.
   Web of Science ®Google Scholar
• Chandrakala S, Deepak K, Revathy G. Anomaly detection in surveillance videos: a thematic taxonomy of deep models, review and performance analysis. Artif Intell Rev. 2022;56:1–50.
   Web of Science ®Google Scholar
• Tu Z, Li H, Xie W, et al. Optical flow for video super-resolution: a survey. Artif Intell Rev. 2022;55(8):6505–6546.
   Web of Science ®Google Scholar