Region-based surveillance video retrieval with effective object representation

ABSTRACT

Retrieving the most relevant video frames that contain the object specified in a given query (query-by-region) remains a challenging task. Two common challenges of region-based retrieval approaches are to accurately extract or segment object(s) and select a proper matching strategy. This paper addresses these problems by proposing a retrieval approach that uses a new region-based matching technique equipped with an effective object representation method. In the first stage, the proposed approach selects the most informative instances of each object that appeared in the video by utilizing an adapted clustering algorithm over the extracted features. In the retrieval stage, the new matching technique returns the most relevant sequences of video by mapping a given region with those identified representative instances of objects based on their similarity scores. The proposed approach is evaluated on standard datasets and the results demonstrate a 31% improvement in the retrieval performance compared to other state-of-the-art methods.

KEYWORDS:

• Region-based retrieval
• query by region
• blob representation
• surveillance video retrieval
• object retrieval
• affinity propagation

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Fereshteh Falah Chamasemani received the B.E. degree in software engineering from the Najaf Abad University, Iran, in 1997, and the M.Sc. degree in Computer Science from the Multimedia University of Malaysia in 2011. She obtained her Ph.D. in computer science in 2017 from the University Putra Malaysia. Her current research interests include content-based image/video retrieval, video/image processing, multimedia databases, pattern recognition, data mining, and machine learning.

Lilly Suriani Affendey is an Associate Professor in the Department of Computer Science, Faculty of Computer Science and Information Technology, University Putra Malaysia. She received her Bachelor of Computer Science in 1991 from the University of Agriculture and in 1994 received her M.Sc. in Computing from the University of Bradford, UK. In 2007, she received her Ph.D. from University Putra Malaysia. Her research interest is in multimedia databases, video content-based retrieval, database security, and data quality.

Norwati Mustapha received her B.Sc. degree in Computer Science from University Putra Malaysia (1991) and MSc degree in Information Systems from University of Leeds (1995). She also obtained her Ph.D. in artificial intelligence from University Putra Malaysia (2005). Dr. Norwati is an active researcher in the area of data mining, web mining, social network and intelligent computing. Now, she is working as Associate Professor at University Putra Malaysia.

Fatimah Khalid obtained the B.Sc. in Computer Science from University Technology Malaysia (UTM) in 1992. During 1993–1995, she worked as a System Analyst at University Kebangsaan Malaysia (UKM) and got her Master’s degree from UKM in 1997. She was involved in teaching at Sal College until 1999 and continued to teach at the University Putra Malaysia from June 1999 until now. She received her Ph.D. in System Science and Management from the National University of Malaysia in 2008. Her research areas are computer vision and image processing, content-based retrieval system and computer graphic applications.

ORCID

Fereshteh Falah Chamasemani http://orcid.org/0000-0002-2188-914X

Lilly Suriani Affendey http://orcid.org/0000-0001-7947-8792

Notes

1. ftp://ftp.cs.rdg.ac.uk/pub/PETS2006/

2. ftp://ftp.cs.rdg.ac.uk/pub/PETS2007/

3. http://wang.ist.psu.edu/docs/related/

Additional information

Funding

This research was supported by the Fundamental Research Grant Scheme from the Malaysian Ministry of Education through grant number FRGS/1/2014/ ICT07/UPM/02/2.