Multi-modal Feature Fusion with Feature Attention for VATEX Captioning Challenge 2020
arXiv:2006.03315
Abstract
This report describes our model for VATEX Captioning Challenge 2020. First, to gather information from multiple domains, we extract motion, appearance, semantic and audio features. Then we design a feature attention module to attend on different feature when decoding. We apply two types of decoders, top-down and X-LAN and ensemble these models to get the final result. The proposed method outperforms official baseline with a significant gap. We achieve 76.0 CIDEr and 50.0 CIDEr on English and Chinese private test set. We rank 2nd on both English and Chinese private test leaderboard.
References in corpus (5)
- Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- Bridging the Gap between Training and Inference for Neural Machine Translation
- Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning
- CNN Architectures for Large-Scale Audio Classification
- Object-aware Aggregation with Bidirectional Temporal Graph for Video Captioning