Open-Ended Multi-Modal Relational Reasoning for Video Question Answering

Abstract

In this paper, we introduce a robotic agent specifically designed to analyze external environments and address participants' questions. The primary focus of this agent is to assist individuals using language-based interactions within video-based scenes. Our proposed method integrates video recognition technology and natural language processing models within the robotic agent. We investigate the crucial factors affecting human-robot interactions by examining pertinent issues arising between participants and robot agents. Methodologically, our experimental findings reveal a positive relationship between trust and interaction efficiency. Furthermore, our model demonstrates a 2\% to 3\% performance enhancement in comparison to other benchmark methods.

2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)

References in corpus (10)

• LSTM: A Search Space Odyssey
• Training language models to follow instructions with human feedback
• ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
• Robust Speech Recognition via Large-Scale Weak Supervision
• Robotic Vision for Human-Robot Interaction and Collaboration: A Survey and Systematic Review
• iPerceive: Applying Common-Sense Reasoning to Multi-Modal Dense Video Captioning and Video Question Answering
• IBERT: Idiom Cloze-style reading comprehension with Attention
• How to Design Sample and Computationally Efficient VQA Models
• SciAnnotate: A Tool for Integrating Weak Labeling Sources for Sequence Labeling
• Question Classification with Deep Contextualized Transformer