Asking the Difficult Questions: Goal-Oriented Visual Question Generation via Intermediate Rewards

Abstract

Despite significant progress in a variety of vision-and-language problems, developing a method capable of asking intelligent, goal-oriented questions about images is proven to be an inscrutable challenge. Towards this end, we propose a Deep Reinforcement Learning framework based on three new intermediate rewards, namely goal-achieved, progressive and informativeness that encourage the generation of succinct questions, which in turn uncover valuable information towards the overall goal. By directly optimizing for questions that work quickly towards fulfilling the overall goal, we avoid the tendency of existing methods to generate long series of insane queries that add little value. We evaluate our model on the GuessWhat?! dataset and show that the resulting questions can help a standard Guesser identify a specific object in an image at a much higher success rate.

References in corpus (11)

• VQA: Visual Question Answering
• Improved Image Captioning via Policy Gradient optimization of SPIDEr
• Learning to Reason: End-to-End Module Networks for Visual Question Answering
• Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering
• Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
• Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
• Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources
• End-to-end optimization of goal-driven and visually grounded dialogue systems
• Visual Question Generation as Dual Task of Visual Question Answering
• GuessWhat?! Visual object discovery through multi-modal dialogue
• Automatic Generation of Grounded Visual Questions

Cited by in corpus (7)

• Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog
• Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat
• Visual Dialogue State Tracking for Question Generation
• Guessing State Tracking for Visual Dialogue
• Learning Better Visual Dialog Agents with Pretrained Visual-Linguistic Representation
• Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation
• Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser