A review of advances in image inpainting research

ABSTRACT

The aim of image inpainting is to fill in damaged areas according to certain rules based on information about the adjacent positions of missing areas and the overall structure of the image, a technique that plays a key role in various tasks in computer vision. With the rapid development of deep learning, researchers have combined it with image inpainting and achieved excellent performance. To gain insight into the techniques involved, this paper summarizes the latest research advances in the field of image inpainting. Firstly, existing classical image inpainting methods are reviewed, and traditional image inpainting methods and their advantages and disadvantages are introduced. Secondly, three classical network models are outlined, and the image inpainting methods are classified into single-stage, multi-stage and a priori condition-guided approaches according to different network types and model structures. Representative algorithms among them are selected and their important technical improvement ideas are analyzed and summarized. Then, the common datasets commonly used in image inpainting tasks and the evaluation metrics used to evaluate inpainting results are introduced. The paper presents a comprehensive summary of the various algorithms in terms of network models and inpainting methods, and selects representative algorithms for quantitative and qualitative comparative analysis. Finally, the future development trends and research directions have prospected, and the current problems of image inpainting are summarized.

KEYWORDS:

• Image inpainting
• convolutional neural network
• generative adversarial network
• variational self-encoder
• evaluation metrics
• dataset

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The project was supported in part by the Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University under Grant VRLAB2023A02, the high-level talent introduction project of Shaanxi Technical College of Finance & Economics Grant 2022KY01 and the Natural Science Basis Research Plan in Shaanxi Province of China under Grant 2023-JC-YB-517.

Notes on contributors

Hong-an Li

Hong-an Li received the M.S. degree in 2009 and the Ph.D. degree in 2014 in computer science and technology from Northwest University, Shaanxi, China. From 2014 to 2023, he was an associate professor of the College of Computer Science and Technology, Xi'an University of Science and Technology. His research interests include computer graphics and computer-aided geometric design, virtual reality and image processing.

Liuqing Hu

Liuqing Hu received the bachelor's degree in software engineering in 2021 from the College of Computer Science and Technology, Xi'an University of Science and Technology. She is currently studying for a master's degree in the College of Computer Science and Technology, Xi'an University of Science and Technology. Her research interests include artificial intelligence, image processing and computer vision applications.

Jun Liu

Jun Liu graduated from the School of Computer Science and Technology, Chengdu University of Technology, for the degree of Bachelor in 1995. He received his M.S. degree in the School of Computer Science and Technology, Northwest University in 2009. Since 2011 he worked toward his Ph.D. degree at Northwest University and received his Ph. D. degree in computer science from Northwest University in 2018. He entered the National-Local Joint Engineering Research Center of Cultural Heritage Digitization in Northwest University, as a researcher from 2016 to 2019. He is a member of China Computer Federation. His current research interests include cultural heritage digitization, pattern recognition and machine learning.

Jing Zhang

Jing Zhang received the M.S. degree in computer application technology from Northwest University of Shaanxi Province, China in 2013 and the Ph.D. degree in 2018. From 2018 to 2023, she served as a lecturer in the School of Computer Science and Technology of Xi'an University of Science and Technology. Her research interests include graphics and image processing, intelligent information processing and signal processing and perception.

Tian Ma

Tian Ma (Member, IEEE) was born in Henan, China, in 1982. He received the B.S. degree in measurement and control technology and instrument, the M.S. degree in software engineering, and the Ph.D. degree in information and communication engineering from Northwestern Polytechnical University, Xi'an, China, in 2003, 2006, and 2011, respectively. Since 2014, he has been an Associated Professor with the College of Computer Science and Technology, Xi'an University of Science and Technology, China. He is currently the author of more than 20 articles and more than ten software copyrights. His research interests include image processing, computer graphics, and 3D simulation and visualization. He has served as the Organizing Committee Co-Chair for the 14th International Conference on Verification and Evaluation of Computer and Communication Systems (VECoS 2020) and a common Reviewer for the IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC).