Microsoft COCO Captions: Data Collection and Evaluation Server

Abstract

In this paper we describe the Microsoft COCO Caption dataset and evaluation server. When completed, the dataset will contain over one and a half million captions describing over 330,000 images. For the training and validation images, five independent human generated captions will be provided. To ensure consistency in evaluation of automatic caption generation algorithms, an evaluation server is used. The evaluation server receives candidate captions and scores them using several popular metrics, including BLEU, METEOR, ROUGE and CIDEr. Instructions for using the evaluation server are provided.

arXiv admin note: text overlap with arXiv:1411.4952

References in corpus (7)

• Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
• Deep Fragment Embeddings for Bidirectional Image Sentence Mapping
• Explain Images with Multimodal Recurrent Neural Networks
• Learning a Recurrent Visual Representation for Image Caption Generation
• From Captions to Visual Concepts and Back
• Phrase-based Image Captioning
• Simple Image Description Generator via a Linear Phrase-Based Approach

Cited by in corpus (356)

• On the Opportunities and Risks of Foundation Models
• VisualBERT: A Simple and Performant Baseline for Vision and Language
• Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
• VQA: Visual Question Answering
• VL-BERT: Pre-training of Generic Visual-Linguistic Representations
• Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)
• Multi-Task Learning with Deep Neural Networks: A Survey
• Exploring Models and Data for Image Question Answering
• SimVLM: Simple Visual Language Model Pretraining with Weak Supervision
• Image Captioning with Semantic Attention
• COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images
• Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion
• A Simple, Fast Diverse Decoding Algorithm for Neural Generation
• Adversarial Text-to-Image Synthesis: A Review
• Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
• Sequence to Sequence -- Video to Text
• Deep Learning Techniques for Future Intelligent Cross-Media Retrieval
• Unifying Vision-and-Language Tasks via Text Generation
• Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge
• Long Text Generation via Adversarial Training with Leaked Information
• Improving Chest X-Ray Report Generation by Leveraging Warm Starting
• ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data
• How Much Can CLIP Benefit Vision-and-Language Tasks?
• Multimodal Machine Learning: A Survey and Taxonomy
• Deep Visual-Semantic Alignments for Generating Image Descriptions
• The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes
• WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
• ImageNet pre-trained models with batch normalization
• AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding
• Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
• Deconfounded Image Captioning: A Causal Retrospect
• Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
• Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training
• CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval
• Learning to Ask: Neural Question Generation for Reading Comprehension
• Unblind Your Apps: Predicting Natural-Language Labels for Mobile GUI Components by Deep Learning
• CPTR: Full Transformer Network for Image Captioning
• Aligning where to see and what to tell: image caption with region-based attention and scene factorization
• DeepSkeleton: Learning Multi-task Scale-associated Deep Side Outputs for Object Skeleton Extraction in Natural Images
• Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
• VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling
• From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities
• From Captions to Visual Concepts and Back
• Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks
• DenseCap: Fully Convolutional Localization Networks for Dense Captioning
• Review Networks for Caption Generation
• Recurrent Topic-Transition GAN for Visual Paragraph Generation
• Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
• Unified Vision-Language Pre-Training for Image Captioning and VQA
• Multilingual Image Description with Neural Sequence Models
• Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space
• Person Search with Natural Language Description
• Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
• Deep Reinforcement Learning-based Image Captioning with Embedding Reward
• VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena
• Attentive Explanations: Justifying Decisions and Pointing to the Evidence
• Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
• Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challenge
• Learning to Decode for Future Success
• Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling
• Emergent Translation in Multi-Agent Communication
• Seeing with Humans: Gaze-Assisted Neural Image Captioning
• Exploring Visual Relationship for Image Captioning
• Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-training
• Video Storytelling: Textual Summaries for Events
• On the effectiveness of task granularity for transfer learning
• Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources
• Visual Question Answering: A Survey of Methods and Datasets
• M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training
• Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering
• Multimedia Semantic Integrity Assessment Using Joint Embedding Of Images And Text
• Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning
• Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination
• Multi-adversarial Faster-RCNN for Unrestricted Object Detection
• Reconstruction Network for Video Captioning
• Detecting Hate Speech in Multi-modal Memes
• VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research
• Gender and Racial Bias in Visual Question Answering Datasets
• 12-in-1: Multi-Task Vision and Language Representation Learning
• A Multimodal Memes Classification: A Survey and Open Research Issues
• Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations
• Several categories of Large Language Models (LLMs): A Short Survey
• What value do explicit high level concepts have in vision to language problems?
• Unifying Two-Stream Encoders with Transformers for Cross-Modal Retrieval
• SentiCap: Generating Image Descriptions with Sentiments
• UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning
• Image Inspired Poetry Generation in XiaoIce
• Are we pretraining it right? Digging deeper into visio-linguistic pretraining
• Envisioning Narrative Intelligence: A Creative Visual Storytelling Anthology
• Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
• Spatio-Temporal Graph for Video Captioning with Knowledge Distillation
• Structured Attention for Unsupervised Dialogue Structure Induction
• Improving Image Captioning with Better Use of Captions
• A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
• A Dataset of Alt Texts from HCI Publications: Analyses and Uses Towards Producing More Descriptive Alt Texts of Data Visualizations in Scientific Papers
• Partially-Supervised Image Captioning
• Spatio-Temporal Attention Models for Grounded Video Captioning
• Embedded Heterogeneous Attention Transformer for Cross-lingual Image Captioning
• SPEECH-COCO: 600k Visually Grounded Spoken Captions Aligned to MSCOCO Data Set
• No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling
• Learning the Best Pooling Strategy for Visual Semantic Embedding
• Attention-Based Multimodal Fusion for Video Description
• Egocentric Image Captioning for Privacy-Preserved Passive Dietary Intake Monitoring
• Self-Training Vision Language BERTs with a Unified Conditional Model
• Object Skeleton Extraction in Natural Images by Fusing Scale-associated Deep Side Outputs
• An Empirical Study of Training End-to-End Vision-and-Language Transformers
• Video Captioning via Hierarchical Reinforcement Learning
• Human-Adversarial Visual Question Answering
• Rethinking the Reference-based Distinctive Image Captioning
• Learning Two-Branch Neural Networks for Image-Text Matching Tasks
• Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation
• Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
• Dataset of Fake News Detection and Fact Verification: A Survey
• Knowledge driven Description Synthesis for Floor Plan Interpretation
• XGPT: Cross-modal Generative Pre-Training for Image Captioning
• SemStyle: Learning to Generate Stylised Image Captions using Unaligned Text
• Contrastive Attention for Automatic Chest X-ray Report Generation
• Consensus-based Sequence Training for Video Captioning
• Semantic Compositional Networks for Visual Captioning
• Recent Advances and Trends in Multimodal Deep Learning: A Review
• In Defense of Grid Features for Visual Question Answering
• Neural Self Talk: Image Understanding via Continuous Questioning and Answering
• Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection
• Open-Vocabulary Object Detection Using Captions
• Generating Diverse and Meaningful Captions
• VIVO: Visual Vocabulary Pre-Training for Novel Object Captioning
• VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions
• An Attempt towards Interpretable Audio-Visual Video Captioning
• Video Captioning with Transferred Semantic Attributes
• Learning to Evaluate Image Captioning
• On Hallucination and Predictive Uncertainty in Conditional Language Generation
• From Deterministic to Generative: Multi-Modal Stochastic RNNs for Video Captioning
• Scaling Up Vision-Language Pre-training for Image Captioning
• BAAI-VANJEE Roadside Dataset: Towards the Connected Automated Vehicle Highway technologies in Challenging Environments of China
• Referenceless Quality Estimation for Natural Language Generation
• Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning
• PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable
• Pre-Trained Models: Past, Present and Future
• Diffusion Based Augmentation for Captioning and Retrieval in Cultural Heritage
• Learning semantic sentence representations from visually grounded language without lexical knowledge
• COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrieval
• Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog
• Attention Neural Network for Trash Detection on Water Channels
• Comprehensive Image Captioning via Scene Graph Decomposition
• A Comparative Review of Recent Few-Shot Object Detection Algorithms
• Linguistically-aware Attention for Reducing the Semantic-Gap in Vision-Language Tasks
• #PraCegoVer: A Large Dataset for Image Captioning in Portuguese
• Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
• Cross-Modal Graph with Meta Concepts for Video Captioning
• Relational Reasoning using Prior Knowledge for Visual Captioning
• A Survey on Biomedical Image Captioning
• Spatio-Temporal Dynamics and Semantic Attribute Enriched Visual Encoding for Video Captioning
• A Semi-supervised Framework for Image Captioning
• Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answering
• Effective End-to-End Vision Language Pretraining with Semantic Visual Loss
• MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual Captioning
• Kaleido-BERT: Vision-Language Pre-training on Fashion Domain
• Convolutional Auto-encoding of Sentence Topics for Image Paragraph Generation
• ClipCap: CLIP Prefix for Image Captioning
• Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers
• Temporal Deformable Convolutional Encoder-Decoder Networks for Video Captioning
• Joint Image Captioning and Question Answering
• Automatic Article Commenting: the Task and Dataset
• On the Evaluation of Vision-and-Language Navigation Instructions
• Non-Autoregressive Image Captioning with Counterfactuals-Critical Multi-Agent Learning
• Putting the Horse Before the Cart:A Generator-Evaluator Framework for Question Generation from Text
• Neuroevolution of Neural Network Architectures Using CoDeepNEAT and Keras
• Dataset vs Reality: Understanding Model Performance from the Perspective of Information Need
• Object-aware Aggregation with Bidirectional Temporal Graph for Video Captioning
• Teacher-Critical Training Strategies for Image Captioning
• Fashion Captioning: Towards Generating Accurate Descriptions with Semantic Rewards
• Deep Multimodal Neural Architecture Search
• Improving CNN-based Planar Object Detection with Geometric Prior Knowledge
• ContCap: A scalable framework for continual image captioning
• Senti-Attend: Image Captioning using Sentiment and Attention
• Unsupervised Image Captioning
• RegionCL: Can Simple Region Swapping Contribute to Contrastive Learning?
• Multi-modal Discriminative Model for Vision-and-Language Navigation
• Improving Neural Question Generation using Answer Separation
• "Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention
• Sensor-Augmented Egocentric-Video Captioning with Dynamic Modal Attention
• Survey of Recent Advances in Visual Question Answering
• LCEval: Learned Composite Metric for Caption Evaluation
• FairyTailor: A Multimodal Generative Framework for Storytelling
• Information Maximizing Visual Question Generation
• Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Network
• VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs
• E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning
• Translation-Enhanced Multilingual Text-to-Image Generation
• Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation
• Dense Captioning with Joint Inference and Visual Context
• ReFormer: The Relational Transformer for Image Captioning
• Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models
• Multimodal Memory Modelling for Video Captioning
• Generating captions without looking beyond objects
• Fine-Grained Image Captioning with Global-Local Discriminative Objective
• Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks
• What Vision-Language Models `See' when they See Scenes
• Fast Sequence Generation with Multi-Agent Reinforcement Learning
• Video-Text Pre-training with Learned Regions
• IDEA: Increasing Text Diversity via Online Multi-Label Recognition for Vision-Language Pre-training
• All-in-One Image-Grounded Conversational Agents
• UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
• Improving Question Generation With to the Point Context
• Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions
• Visual News: Benchmark and Challenges in News Image Captioning
• Reflective Decoding Network for Image Captioning
• Learning Object Detection from Captions via Textual Scene Attributes
• Generating Question Relevant Captions to Aid Visual Question Answering
• End-to-end Image Captioning Exploits Multimodal Distributional Similarity
• Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling
• Cap2Det: Learning to Amplify Weak Caption Supervision for Object Detection
• LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
• Multimodal Generative Models for Compositional Representation Learning
• End-to-End Video Captioning with Multitask Reinforcement Learning
• Visual Referring Expression Recognition: What Do Systems Actually Learn?
• Recurrent Image Captioner: Describing Images with Spatial-Invariant Transformation and Attention Filtering
• Causal Attention for Vision-Language Tasks
• Better Captioning with Sequence-Level Exploration
• Multimodal Learning for Hateful Memes Detection
• Reconstruct and Represent Video Contents for Captioning via Reinforcement Learning
• CoTK: An Open-Source Toolkit for Fast Development and Fair Evaluation of Text Generation
• Semantic-aware Image Deblurring
• Journalistic Guidelines Aware News Image Captioning
• In-Home Daily-Life Captioning Using Radio Signals
• Human-like Controllable Image Captioning with Verb-specific Semantic Roles
• Probabilistic Embeddings for Cross-Modal Retrieval
• Identifying Well-formed Natural Language Questions
• Semi-supervised Multimodal Representation Learning through a Global Workspace
• Batch-normalized Recurrent Highway Networks
• Self-supervised pre-training and contrastive representation learning for multiple-choice video QA
• ARAML: A Stable Adversarial Training Framework for Text Generation
• Visually Grounded Continual Learning of Compositional Phrases
• Learning to discover and localize visual objects with open vocabulary
• KANDINSKYPatterns -- An experimental exploration environment for Pattern Analysis and Machine Intelligence
• Probing Image-Language Transformers for Verb Understanding
• Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasets
• ROSITA: Enhancing Vision-and-Language Semantic Alignments via Cross- and Intra-modal Knowledge Integration
• Visual Fashion-Product Search at SK Planet
• A Picture May Be Worth a Hundred Words for Visual Question Answering
• Generating Diverse and Informative Natural Language Fashion Feedback
• Bringing back simplicity and lightliness into neural image captioning
• Image Captioning using Deep Stacked LSTMs, Contextual Word Embeddings and Data Augmentation
• Unpaired Image-to-Speech Synthesis with Multimodal Information Bottleneck
• VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer
• Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization
• Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements
• Learning to Represent Image and Text with Denotation Graph
• Response-based Distillation for Incremental Object Detection
• B-SCST: Bayesian Self-Critical Sequence Training for Image Captioning
• Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation
• Exploring and Distilling Cross-Modal Information for Image Captioning
• LAMP: Label Augmented Multimodal Pretraining
• Phrase Localization Without Paired Training Examples
• Neural Text Generation with Artificial Negative Examples
• Attend More Times for Image Captioning
• DeepFlux for Skeletons in the Wild
• Learning Comment Generation by Leveraging User-Generated Data
• Fast and Simple Mixture of Softmaxes with BPE and Hybrid-LightRNN for Language Generation
• Neural Aesthetic Image Reviewer
• Structure-Aware Generation Network for Recipe Generation from Images
• Ladder Loss for Coherent Visual-Semantic Embedding
• Understanding Contexts Inside Robot and Human Manipulation Tasks through a Vision-Language Model and Ontology System in a Video Stream
• M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention
• Exploring Semantic Relationships for Unpaired Image Captioning
• Survey: Transformer based Video-Language Pre-training
• Structural and Functional Decomposition for Personality Image Captioning in a Communication Game
• Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge
• Video Question Generation via Cross-Modal Self-Attention Networks Learning
• Improving Reinforcement Learning Based Image Captioning with Natural Language Prior
• Improving Image Captioning by Leveraging Knowledge Graphs
• Face-Cap: Image Captioning using Facial Expression Analysis
• Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection
• Geometry-Entangled Visual Semantic Transformer for Image Captioning
• simNet: Stepwise Image-Topic Merging Network for Generating Detailed and Comprehensive Image Captions
• Evaluating Text-to-Image Matching using Binary Image Selection (BISON)
• Semi-Autoregressive Transformer for Image Captioning
• DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation
• The Multi-Modal Video Reasoning and Analyzing Competition
• Collaborative Training of GANs in Continuous and Discrete Spaces for Text Generation
• Paying Attention to Descriptions Generated by Image Captioning Models
• Empirical Autopsy of Deep Video Captioning Frameworks
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation
• Connecting What to Say With Where to Look by Modeling Human Attention Traces
• Enhance Multimodal Model Performance with Data Augmentation: Facebook Hateful Meme Challenge Solution
• ProMask: Probability Mask for Skeleton Detection
• Diverse and Styled Image Captioning Using SVD-Based Mixture of Recurrent Experts
• Don't Change Me! User-Controllable Selective Paraphrase Generation
• Data augmentation to improve robustness of image captioning solutions
• R${}^3$Net:Relation-embedded Representation Reconstruction Network for Change Captioning
• Visual Reasoning with Natural Language
• SPICE: Semantic Propositional Image Caption Evaluation
• Semantic Grouping Network for Video Captioning
• Semi-Autoregressive Image Captioning
• Argus: Leveraging Multiview Images for Improved 3-D Scene Understanding With Large Language Models
• Does Vision-and-Language Pretraining Improve Lexical Grounding?
• Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learning
• SparseGAN: Sparse Generative Adversarial Network for Text Generation
• Generating Pertinent and Diversified Comments with Topic-aware Pointer-Generator Networks
• Chop Chop BERT: Visual Question Answering by Chopping VisualBERT's Heads
• Video Captioning Using Weak Annotation
• Math Word Problem Generation with Mathematical Consistency and Problem Context Constraints
• SMURF: SeMantic and linguistic UndeRstanding Fusion for Caption Evaluation via Typicality Analysis
• On Architectures for Including Visual Information in Neural Language Models for Image Description
• Goal-driven text descriptions for images
• X-modaler: A Versatile and High-performance Codebase for Cross-modal Analytics
• Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning
• Less is More: Generating Grounded Navigation Instructions from Landmarks
• Unsupervised Bilingual Lexicon Induction from Mono-lingual Multimodal Data
• Know What You Don't Know: Modeling a Pragmatic Speaker that Refers to Objects of Unknown Categories
• Vatex Video Captioning Challenge 2020: Multi-View Features and Hybrid Reward Strategies for Video Captioning
• Achieving Human Parity on Visual Question Answering
• Learning Compact Reward for Image Captioning
• "Wikily" Supervised Neural Translation Tailored to Cross-Lingual Tasks
• Bidirectional Multirate Reconstruction for Temporal Modeling in Videos
• Survey of Visual-Semantic Embedding Methods for Zero-Shot Image Retrieval
• Middle-Out Decoding
• Adversarial Machine Learning in Text Analysis and Generation
• GEM: A General Evaluation Benchmark for Multimodal Tasks
• The Long-Short Story of Movie Description
• Hidden State Guidance: Improving Image Captioning using An Image Conditioned Autoencoder
• Bridging Visual Perception with Contextual Semantics for Understanding Robot Manipulation Tasks
• Improving Calibration in Deep Metric Learning With Cross-Example Softmax
• Interpretation of 3D CNNs for Brain MRI Data Classification
• Bootstrapping Disjoint Datasets for Multilingual Multimodal Representation Learning
• DESCGEN: A Distantly Supervised Dataset for Generating Abstractive Entity Descriptions
• SciXGen: A Scientific Paper Dataset for Context-Aware Text Generation
• Semantic sentence similarity: size does not always matter
• Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text
• Object-Centric Unsupervised Image Captioning
• O2NA: An Object-Oriented Non-Autoregressive Approach for Controllable Video Captioning
• Stack-VS: Stacked Visual-Semantic Attention for Image Caption Generation
• Learning Joint Representations of Videos and Sentences with Web Image Search
• Partially-Supervised Novel Object Captioning Leveraging Context from Paired Data
• Scene-based Factored Attention for Image Captioning
• Counterfactual Maximum Likelihood Estimation for Training Deep Networks
• Video Captioning in Compressed Video
• VideoMCC: a New Benchmark for Video Comprehension
• Adversarial Evaluation of Multimodal Models under Realistic Gray Box Assumption
• Transductive Visual Verb Sense Disambiguation
• Caption Enriched Samples for Improving Hateful Memes Detection
• Distinctive-attribute Extraction for Image Captioning
• Approximate Distribution Matching for Sequence-to-Sequence Learning
• Dual Reinforcement-Based Specification Generation for Image De-Rendering
• Spoken ObjectNet: A Bias-Controlled Spoken Caption Dataset
• Analysis of Convolutional Decoder for Image Caption Generation
• Visual Understanding and Narration: A Deeper Understanding and Explanation of Visual Scenes
• Relationship-based Neural Baby Talk
• Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning
• Understanding in Artificial Intelligence
• Panoptic Narrative Grounding
• SwAMP: Swapped Assignment of Multi-Modal Pairs for Cross-Modal Retrieval
• Cross-Modal Retrieval Augmentation for Multi-Modal Classification
• A Revised Generative Evaluation of Visual Dialogue
• Extracting and filtering paraphrases by bridging natural language inference and paraphrasing
• A Question Type Driven and Copy Loss Enhanced Frameworkfor Answer-Agnostic Neural Question Generation