Performance Evaluation of Shot Boundary Detection Metrics in the Presence of Object and Camera Motion

Abstract

Partitioning a video into shots is an important step for video indexing. We have presented the performance of various traditional metrics that are generally used to detect shot boundaries. In this paper, we evaluated shot boundary detection metrics, such as likelihood ratio and color ratio histogram in Red Green Blue (RGB) and Hue Saturation, Value (HSV) color space for three different action and thriller movies. These movies consist of large number of frames with object and camera motion. The pixel difference and Chi-square shot boundary detection metrics in Luma and Chrominance Components (YUV) color space has been tested for Ave different movies. The results were evaluated in terms of Recall, Precision, and F1 measure for all these movies. It has been observed that these results are affected by the disturbance due to the motion in the consecutive frames. The false positives and miss detection of shot boundaries in all the tested metrics are due to fast camera and object motion. An algorithm has been proposed for shot boundary detection by using dual tree complex wavelet transform in the presence of motion. Performance comparison of the proposed algorithm with the traditional metrics validates its effectiveness in terms of improved Recall, Precision, and F1 score.

Keywords:

• Dual tree complex wavelet transform
• Shot boundary detection
• Traditional metrics
• Recall
• Precision

Additional information

Notes on contributors

Krishna K. Warhade

Krishna K. Warhade received the Bachelor of Engineering in Electronics in 1995 and Master of Engineering in Instrumentation in 1999 both from Shri Guru Gobind Singhaji Institute of Engineering and Technology, Nanded, and Ph. D. in November 2010 from the Department of Electrical Engineering, Indian Institute of Technology Bombay, India. He has 16 years of experience in teaching and research. He is currently working as a Professor in the Department of Electronics Engineering, Lokmanya Tilak College of Engineering, Navi Mumbai, India. His research interests are in the area of signal processing, image processing, video segmentation, video retrieval and wavelets. E-mail: [email protected]

Shabbier N. Merchant

Shabbier N. Merchant is a Professor in Department of Electrical Engineering, Indian Institute of Technology, Bombay. He has received his B. Tech, M. Tech, and PhD degrees all from Department of Electrical Engineering, Indian Institute of Technology, Bombay, India. He is a Fellow of IETE (The Institution of Electronic & Telecommunication Engineers). He is a recipient of 10^th IETE Prof. S.V.C. Aiya Memorial Award for his contribution in the field of detection and tracking. He is also a recipient of 9^th IETE S.V.C. Aiya Memorial Award for Excellence in Telecom Education. He has more than 25 years experience in teaching and research. His noteworthy contributions have been in solving state of the art signal and image processing problems faced by Indian defense. E-mail: [email protected]

U. B. Desai

U. B. Desai received his PhD degree in Electrical Engineering from the Johns Hopkins University, Baltimore, USA, in 1979. From 1979 to 1984, he was as Assistant Professor in the Electrical Engineering Department at Washington State University, Pullman, WA, USA, and Associate Professor at the same place from 1984 to 1987. From 1987 onwards, he was Professor in the Electrical Engineering Department at the Indian Institute of Technology, Bombay. He is a Fellow of Indian National Science Academy (INSA) and Indian National Academy of Engineering (INAE). From July 2002 to June 2004, he was the Director of HP-IITM R and D Lab, at IIT-Madras. Since 2009, Prof. Desai has taken charge as the first Director of Indian Institute of Technology, Hyderabad, India. E-mail: [email protected]