6.2k citations · 14.2k across the 134 of their papers we have counts for
15 papers · 1 filter
nocaps: novel object captioning at scale
Harsh Agrawal, Karan Desai, Yufei Wang +7
Image captioning models have achieved impressive results on datasets containing limited visual concepts and large amounts of paired image-caption training data. However, if these m…
Fabrik: An Online Collaborative Neural Network Editor
Utsav Garg, Viraj Prabhu, Deshraj Yadav +3
We present Fabrik, an online neural network editor that provides tools to visualize, edit, and share neural networks from within a browser. Fabrik provides a simple and intuitive G…
TarMAC: Targeted Multi-Agent Communication
Abhishek Das, Théophile Gervet, Joshua Romoff +4
We propose a targeted communication architecture for multi-agent reinforcement learning, where agents learn both what messages to send and whom to address them to while performing…
Neural Modular Control for Embodied Question Answering
Abhishek Das, Georgia Gkioxari, Stefan Lee +2
We present a modular approach for learning policies for navigation over long planning horizons from language input. Our hierarchical policy operates at multiple timescales, where t…
Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition
Jianwei Yang, Jiasen Lu, Stefan Lee +2
In an open-world setting, it is inevitable that an intelligent agent (e.g., a robot) will encounter visual objects, attributes or relationships it does not recognize. In this work,…
Visual Coreference Resolution in Visual Dialog using Neural Module Networks
Satwik Kottur, José M. F. Moura, Devi Parikh +2
Visual dialog entails answering a series of questions grounded in an image, using dialog history as context. In addition to the challenges found in visual question answering (VQA),…
Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance
Ramprasaath R. Selvaraju, Prithvijit Chattopadhyay, Mohamed Elhoseiny +4
Individual neurons in convolutional neural networks supervised for image-level classification tasks have been shown to implicitly learn semantically meaningful concepts ranging fro…
Graph R-CNN for Scene Graph Generation
Jianwei Yang, Jiasen Lu, Stefan Lee +2
We propose a novel scene graph generation model called Graph R-CNN, that is both effective and efficient at detecting objects and their relations in images. Our model contains a Re…
Pythia v0.1: the Winning Entry to the VQA Challenge 2018
Yu Jiang, Vivek Natarajan, Xinlei Chen +3
This document describes Pythia v0.1, the winning entry from Facebook AI Research (FAIR)'s A-STAR team to the VQA Challenge 2018. Our starting point is a modular re-implementation o…
Talk the Walk: Navigating New York City through Grounded Dialogue
Harm de Vries, Kurt Shuster, Dhruv Batra +3
We introduce "Talk The Walk", the first large-scale dialogue dataset grounded in action and perception. The task involves two agents (a "guide" and a "tourist") that communicate vi…
End-to-End Audio Visual Scene-Aware Dialog using Multimodal Attention-Based Video Features
Chiori Hori, Huda Alamri, Jue Wang +10
Dialog systems need to understand dynamic visual scenes in order to have conversations with users about the objects and events around them. Scene-aware dialog systems for real-worl…
Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations
Ashwin Kalyan, Stefan Lee, Anitha Kannan +1
Many structured prediction problems (particularly in vision and language domains) are ambiguous, with multiple outputs being correct for an input - e.g. there are many ways of desc…
Audio Visual Scene-Aware Dialog (AVSD) Challenge at DSTC7
Huda Alamri, Vincent Cartillier, Raphael Gontijo Lopes +8
Scene-aware dialog systems will be able to have conversations with users about the objects and events around them. Progress on such systems can be made by integrating state-of-the-…
Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples
Ashwin Kalyan, Abhishek Mohta, Oleksandr Polozov +3
Synthesizing user-intended programs from a small number of input-output examples is a challenging problem with several important applications like spreadsheet manipulation, data wr…
Neural Baby Talk
Jiasen Lu, Jianwei Yang, Dhruv Batra +1
We introduce a novel framework for image captioning that can produce natural language explicitly grounded in entities that object detectors find in the image. Our approach reconcil…