Fused features for no reference image quality assessment

ABSTRACT

Computer vision algorithms for image classification, object detection or segmentation tasks require high-quality images for accurate predictions. With quality degraded images, the algorithms may not detect objects properly or can lead to false detections and wrong classifications. To avoid such problems, we developed an image quality estimator to assess the quality of input video feeds for deciding whether to use the frame or need enhancement or discard it altogether. Traditional algorithms fall short of neural networks in terms of accuracy when estimating image quality. Despite remarkable progress in image processing tasks, Neural networks have not been used widely for image quality estimation. The reason might be the lack of large datasets for image quality assessment, which are subjective, expensive, and time-consuming to create. We propose a Fused Feature Image Quality Estimator (FIQE), which uses both traditional handcrafted and convolutional features to estimate the image quality. The performance evaluation results are compared against the state-of-the-art methods and obtained LCC score of 0.956 and SROCC of 0.955 on the LIVE Dataset and 0.922 LCC and 0.904 SROCC on the TID 2013 Dataset for the regressor model. FIQE classifier model achieved 71.06% top-1 accuracy and 97.46% top-2 accuracy on the LIVE dataset.

KEYWORDS:

• Fused CNN handcrafted features
• image quality assessment
• quality regression module
• no reference image quality estimation

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was funded by the Government of India through DST-ICPS (Department of Science and Technology – Interdisciplinary Cyber-Physical Systems) project No. DST/ICPS/CPS-Individual/2018/392 titled ‘Energy efficient cyber security implementations for Internet of Things’. This research work is supported by Vuelogix Technologies Pvt Ltd, Confederation of Indian Industry (CII) and DST-SERB, Govt of India through Prime Minister's Fellowship for Doctoral Research 2020.

Notes on contributors

Ajai John Chemmanam

Ajai John Chemmanam currently a Research scholar in Artificial Intelligence and Machine Learning at the Department of Electronics, CUSAT, Kerala, India. His primary research area is human behaviour analysis using computer vision algorithms. He obtained his Masters and Bachelors degree from Cochin University of Science and Technology, Kerala, India. He is an active contributor to various open-source projects. He has more than five years of industrial experience in AI and ML. His research interests include Image and Video analytics using Deep Learning, Security threats and defences for Neural Networks.

Shahanaz N

Shahanaz N has completed her Mtech in Electronics and Communication Engineering from CUSAT. Her post graduation specialization was in VLSI and Embedded Systems. She did her Btech in Electronics and Communication Engineering from Cochin University. She has been an intern with the NEST cyber campus working on Embedded Systems and VHDL. She has also done an internship at Doordarshan in the field of telecommunications. Her areas of interest are VLSI Design, Machine Learning and Embedded systems.

Bijoy A Jose

Bijoy A Jose currently working as an Assistant Professor in the Department of Electronics, CUSAT, Kerala, India. He has received his B.Tech from the School of Engineering CUSAT and M.S from the State University of New York. He has been an intern with IBM, New York and Intel Corporation, Illinois, during his graduate studies. He received his PhD from Virginia Tech and worked at Intel Corporation in California and Bangalore for four years. He received the Early Career Research Award from the Department of Science and Technology, Govt. of India, in 2016. He is the Principal Investigator to multiple funded projects from DST and IEEE. His areas of interest include Cyber security, the Internet of Things and Cyber-Physical Systems.