Defect detection with ego-noise reduction based on multimodal information in UAV hammering inspection

Abstract

In this paper, we introduce a novel approach for defect detection in hammering inspections using Unmanned Aerial Vehicles (UAVs). Despite the promising application of UAVs for inspecting hard-to-reach structures, like bridges, their efficiency is often compromised by the significant ego noise produced by their motor-propeller systems. This noise complicates the discrimination between healthy and defective hammering sounds. In previous research, methods to improve robustness through supervised learning have been proposed; however, these methods require the labeling of hammering sounds by skilled inspectors to train the discrimination model. To overcome this problem, we propose an ego-noise reduction method based on propeller vibrations. By reducing ego noise and thereby making the characteristics of hammering sound more dramatically clear, we enable unsupervised defect detection amidst ego noise. Our experiments with concrete specimens demonstrate that our technique achieves defect detection with an accuracy on par with the supervised method. The proposed method proves especially beneficial for hammering inspections, in which the domain gap–the variability in acoustic signatures of hammering sounds caused by differences in concrete mix ratios and curing conditions from one site to another–presents a significant challenge. Our approach effectively adapts to these variations, ensuring reliable defect detection across diverse construction environments.

GRAPHICAL ABSTRACT

Keywords:

• Infrastructure inspection
• hammering test
• field robotics
• multimodal learning
• sound sensing

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported in part by JSPS KAKENHI under Grant Number JP21K17829.

Notes on contributors

Koki Shoda

Koki Shoda received a B.E. degree from the Department of Precision Engineering, the University of Tokyo, Japan in 2024. From 2024, he is a master's student in the Department of Human Environment Studies, The University of Tokyo. His research interests include signal processing and machine learning. He is a member of SICE and RSJ.

Jun Younes Louhi Kasahara

Jun Younes Louhi Kasahara received B.E. and Diplôme d'Ingénieur degrees from the Ecole Centrale de Lyon in 2013 and 2016, respectively. He received his Ph.D. degree from The University of Tokyo in 2019. From 2019 to 2020, he was a specially appointed researcher at The University of Tokyo. From 2020 to 2022, he was a project assistant professor at The University of Tokyo. Since 2022, he has been a project lecturer at The University of Tokyo. His research interests are machine learning, algorithms, signal processing, and automation. He is a member of IEEE, JSPE, JSME and RSJ.

Hajime Asama

Hajime Asama received the M.S. and Dr.Eng. degrees from The University of Tokyo (UTokyo), in 1984 and 1989, respectively. He was with Riken, Japan from 1986 to 2002. He became a Professor with the Research into Artifacts, Center for Engineering (RACE) of UTokyo in 2002. He was a Professor with the School of Engineering of Utokyo from 2009 to 2024, and the Director of RACE from 2019 to 2023. Currently, he is a project professor at Tokyo College, UTokyo. His main research interests include service robotics, distributed autonomous robotic systems, embodied brain science systems, and cognitive ergonomics. He was an AdCom Member of the IEEE Robotics and Automation Society, from 2007 to 2009. He has been a Council Member of the Science Council of Japan from 2017 to 2023. He is a fellow of IEEE, JSME, RSJ and SICE. He received the JSME Award (Technical Achievement) in 2018, etc. He was the Vice President of RSJ from 2011 to 2012, the Vice President of JSME in 2023, and the President of IFAC from 2020 to 2023.

Qi An

Qi An received his B.E., M.E. and Ph. D. degrees in Engineering from The University of Tokyo in 2009, 2011 and 2014, respectively. From 2012, he is a JSPS Research Fellowship for Young Scientists (DC1). From 2015 to 2020, he was an Assistant Professor of The University of Tokyo. From 2020 to 2022, he was an Associate Professor of Kyushu University. From 2022, he is an Associate Professor of The University of Tokyo. He is a member of IEEE, JSPE, RSJ, JSME, SICE and LSE.

Atsushi Yamashita

Atsushi Yamashita received B.E., M.E., and Ph.D. degrees from the Faculty of Engineering, the University of Tokyo in 1996, 1998, and 2001, respectively. From 2001 to 2008, he was an Assistant Professor at Shizuoka University. From 2006 to 2007, he was a Visiting Associate at California Institute of Technology. From 2008 to 2011, he was an Associate Professor at Shizuoka University. From 2011 to 2022, he was an Associate Professor at The University of Tokyo. From 2022, he is a Professor of The University of Tokyo. His research interests are robot vision, image processing, and intelligent robot systems. He is a member of ACM, IEEE, JSPE, RSJ, IEICE, JSAE, JSCE, JSME, IEEJ, IPSJ, ITE, SICE, and Society for Serviceology.