Deep learning-based Video Captioning Technique using Transformer

Abstract

Automatic Video captioning is the process by which a meaningful natural language sentence description is generated for a given video. Video understanding has got several applications in the field of automatic object identification, real-time calamity detection, navigation for visually impaired people, autonomous driving, human robot interaction and many more. The process of video captioning is a tedious task for machines, and it needs a collaborated effort of CV-Computer Vision techniques and NLP-Natural Language Processing techniques. In the last decade, the technologies that were used to perform this task includes CNN, RNN, LSTM, GRU and its variants. Recently the focus has been shifted to Transformer networks which sounds more promising than any of these. This work proposes a transformer-based Video captioning architecture, and the evaluation has been made over standard dataset with metrics and is found to perform superior to existing methods.