Creativity: Generating Diverse Questions using Variational Autoencoders

Abstract

Generating diverse questions for given images is an important task for computational education, entertainment and AI assistants. Different from many conventional prediction techniques is the need for algorithms to generate a diverse set of plausible questions, which we refer to as "creativity". In this paper we propose a creative algorithm for visual question generation which combines the advantages of variational autoencoders with long short-term memory networks. We demonstrate that our framework is able to generate a large set of varying questions given a single input image.

Accepted to CVPR 2017

References in corpus (14)

• Sequence to Sequence Learning with Neural Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Improved Techniques for Training GANs
• Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
• Hierarchical Question-Image Co-Attention for Visual Question Answering
• DRAW: A Recurrent Neural Network For Image Generation
• Dynamic Memory Networks for Visual and Textual Question Answering
• Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)
• Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
• Fully Connected Deep Structured Networks
• Multimodal Residual Learning for Visual QA
• Deep Visual-Semantic Alignments for Generating Image Descriptions
• Visual Madlibs: Fill in the blank Image Generation and Question Answering
• CIDEr: Consensus-based Image Description Evaluation

Cited by in corpus (12)

• Multimodal Intelligence: Representation Learning, Information Fusion, and Applications
• Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question Answering
• High-Order Attention Models for Visual Question Answering
• Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
• Visual Question Generation as Dual Task of Visual Question Answering
• Improving Missing Data Imputation with Deep Generative Models
• FlipDial: A Generative Model for Two-Way Visual Dialogue
• Towards a Better Metric for Evaluating Question Generation Systems
• Structural Consistency and Controllability for Diverse Colorization
• Group-based Distinctive Image Captioning with Memory Attention
• Generating Diverse Translation by Manipulating Multi-Head Attention
• Dynamic Attention Networks for Task Oriented Grounding