Improving Video Stabilization Using Multi-Resolution MSER Features

ABSTRACT

In this paper, we investigate the application of multi-resolution maximally stable extremal region (MSER) features for improving the video stabilization performance. MSER features have been used for many computer vision applications like wide baseline stereo, object recognition, video object tracking, and video stabilization with very good results as compared to other features like scale invariant feature transform (SIFT) and Kanade Lucas Tomasi (KLT). However, a limitation of the MSER feature in the stabilization application was observed when the input video frames were severely blurred. The same limitation was also observed when other features like KLT and SIFT were utilized under blurring conditions. In this paper we propose to overcome this drawback for video stabilization application by utilizing MSERs which are extracted and matched in a scale pyramid fashion instead of the MSER features detected and matched on a single image resolution. The duplicate MSERs resulting due to the pyramid style detection are removed followed by MSER feature matching for establishing correspondence between video frames to estimate the global motion parameters. Once the global motion parameters are estimated, the accumulated transformation is smoothed followed by motion compensation to construct the stabilized frame. Comparative analysis with state-of-the-art stabilization methods shows improvement in stabilization performance as well as robustness to blurring degradations. The proposed method can easily be ported to other feature detectors like KLT and SIFT thereby making the proposed method generic to any feature detector.

Keywords:

• Feature matching
• Global motion estimation
• Motion compensation
• MSER
• Multi-resolution analysis
• Video stabilization

ACKNOWLEDGEMENTS

We would like to thank Prof. Subhashis Banerjee of IIT Delhi and Prof. Uma Mudenagudi of BVBCET Hubli for their valuable feedback on our work. We would also like to thank the anonymous reviewers as well as the editor who coordinated the review of our paper.

Additional information

Notes on contributors

Manish Okade

Manish Okade was born in Hubli, Karnataka, India, in 1980. He received his BE degree in electronics and communication engineering from BVB College of Engineering and Technology, Hubli, MTech degree in automation and computer vision from Indian Institute of Technology (IIT), Kharagpur, and PhD degree in computer vision and image processing from IIT, Kharagpur. He was awarded with the IBM “The Great Mind Challenge Award” in the year 2008 for the project OSPEDALE. He was a senior lecturer in the Dept. of Computer Science and Engineering at BVB College of Engineering and Technology, Hubli, from 2006 to 2009. Currently he is working as an assistant professor in the Dept. of Electronics and Communication Engineering, National Institute of Technology (NIT), Rourkela, India. His interests include image and video processing, pattern recognition, assistive technologies, and road safety technologies.

E-mail: [email protected]

Prabir Kumar Biswas

Prabir K. Biswas received the BTech degree (with honors) in electronics and electrical communication engineering, the MTech. degree in automation and control engineering, and the PhD degree in computer vision from the Department of Electronics and Electrical Communication Engineering, IIT, Kharagpur, India, in 1985, 1989, and 1991, respectively. From 1985 to 1987, he was with Bharat Electronics Ltd, Ghaziabad, India, as a deputy engineer. Since 1991, he has been working as a faculty member in the Department of Electronics and Electrical Communication Engineering, IIT, where he is currently a professor. He has several video lectures hosted on NPTEL which is India's premier knowledge hub supported by Ministry of HRD, India. His interests include pattern recognition, image processing, and operating systems.

E-mail: [email protected]