Caption positioning structure for hard of hearing people using deep learning method

Abstract

Captioning for hard of hearing people is one of the priceless assistances to watch and understand various communication medium like TV, movies and others. The time presently appears to be ideal to respond to urgent inquiries and recognize the addressees and their needs/inclinations. Considering in terms of deafened, they will not understand who is asking the question and who is answering them back. This is the major drawback where a person with hearing disability will be unsure of the scene and about the whole movie. To overcome from this drawback, we propose a comprehensive captioning framework which is designed to extend the availability and interaction of the hard of hearing and hearing impaired: A full-fledged capture positioning structure that supports any multimedia file and a sensible inscription region movement method that lets the subtitle to be fittingly chosen the screen, considering the perceptible bit of the video scene and the dynamic/alive spokesperson perceived. The proposed framework for captioning is developed using Open CV and R-CNN. It consists of three modules as: 1. Audio Processing, 2. Video Processing, 3. Subtitle Positioning. The goal of this project is to give a better visual experience for the deafened and hard of hearing people.

Subject Classification:

• 68W99

Keywords:

• Regions with convolutional neural networks (R-CNN)
• Region of interest (ROI)