978 citations · 7k across the 148 of their papers we have counts for
22 papers · 1 filter
Look Before you Speak: Visually Contextualized Utterances
Paul Hongsuck Seo, Arsha Nagrani, Cordelia Schmid
While most conversational AI systems focus on textual dialogue only, conditioning utterances on visual context (when it's available) can lead to more realistic conversations. Unfor…
Image Matching with Scale Adjustment
Yves Dufournaud, Cordelia Schmid, Radu Horaud
In this paper we address the problem of matching two images with two different resolutions: a high-resolution image and a low-resolution one. The difference in resolution between t…
Just Ask: Learning to Answer Questions from Millions of Narrated Videos
Antoine Yang, Antoine Miech, Josef Sivic +2
Recent methods for visual question answering rely on large-scale annotated datasets. Manual annotation of questions and answers for videos, however, is tedious, expensive and preve…
TNT: Target-driveN Trajectory Prediction
Hang Zhao, Jiyang Gao, Tian Lan +9
Predicting the future behavior of moving agents is essential for real world applications. It is challenging as the intent of the agent and the corresponding behavior is unknown and…
The End-of-End-to-End: A Video Understanding Pentathlon Challenge (2020)
Samuel Albanie, Yang Liu, Arsha Nagrani +18
We present a new video understanding pentathlon challenge, an open competition held in conjunction with the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020.…
Learning Obstacle Representations for Neural Motion Planning
Robin Strudel, Ricardo Garcia, Justin Carpentier +3
Motion planning and obstacle avoidance is a key challenge in robotics applications. While previous work succeeds to provide excellent solutions for known environments, sensor-based…
Uncertainty-Aware Weakly Supervised Action Detection from Untrimmed Videos
Anurag Arnab, Chen Sun, Arsha Nagrani +1
Despite the recent advances in video classification, progress in spatio-temporal action recognition has lagged behind. A major contributing factor has been the prohibitive cost of…
Multi-modal Transformer for Video Retrieval
Valentin Gabeur, Chen Sun, Karteek Alahari +1
The task of retrieving video content relevant to natural language queries plays a critical role in effectively handling internet-scale datasets. Most of the existing methods for th…
Learning Video Representations from Textual Web Supervision
Jonathan C. Stroud, Zhichao Lu, Chen Sun +4
Videos on the Internet are paired with pieces of text, such as titles and descriptions. This text typically describes the most important content in the video, such as the objects i…
Unsupervised Learning of Video Representations via Dense Trajectory Clustering
Pavel Tokmakov, Martial Hebert, Cordelia Schmid
This paper addresses the task of unsupervised learning of representations for action recognition in videos. Previous works proposed to utilize future prediction, or other domain-sp…
Consistency Guided Scene Flow Estimation
Yuhua Chen, Luc Van Gool, Cordelia Schmid +1
Consistency Guided Scene Flow Estimation (CGSF) is a self-supervised framework for the joint reconstruction of 3D scene structure and motion from stereo video. The model takes two…
TAO: A Large-Scale Benchmark for Tracking Any Object
Achal Dave, Tarasha Khurana, Pavel Tokmakov +2
For many years, multi-object tracking benchmarks have focused on a handful of categories. Motivated primarily by surveillance and self-driving applications, these datasets provide…
VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation
Jiyang Gao, Chen Sun, Hang Zhao +4
Behavior prediction in dynamic, multi-agent systems is an important problem in the context of self-driving cars, due to the complex representations and interactions of road compone…
What Makes for Good Views for Contrastive Learning?
Yonglong Tian, Chen Sun, Ben Poole +3
Contrastive learning between multiple views of the data has recently achieved state of the art performance in the field of self-supervised representation learning. Despite its succ…
Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction
Yana Hasson, Bugra Tekin, Federica Bogo +3
Modeling hand-object manipulations is essential for understanding how humans interact with their environment. While of practical importance, estimating the pose of hands and object…
Learning visual policies for building 3D shape categories
Alexander Pashevich, Igor Kalevatykh, Ivan Laptev +1
Manipulation and assembly tasks require non-trivial planning of actions depending on the environment and the final goal. Previous work in this domain often assembles particular ins…
Memory-Efficient Incremental Learning Through Feature Adaptation
Ahmet Iscen, Jeffrey Zhang, Svetlana Lazebnik +1
We introduce an approach for incremental learning that preserves feature descriptors of training images from previously learned classes, instead of the images themselves, unlike mo…
Speech2Action: Cross-modal Supervision for Action Recognition
Arsha Nagrani, Chen Sun, David Ross +3
Is it possible to guess human action from dialogue alone? In this work we investigate the link between spoken words and actions in movies. We note that movie screenplays describe a…
Beyond the Camera: Neural Networks in World Coordinates
Gunnar A. Sigurdsson, Abhinav Gupta, Cordelia Schmid +1
Eye movement and strategic placement of the visual field onto the retina, gives animals increased resolution of the scene and suppresses distracting information. This fundamental s…
Selecting Relevant Features from a Multi-domain Representation for Few-shot Classification
Nikita Dvornik, Cordelia Schmid, Julien Mairal
Popular approaches for few-shot classification consist of first learning a generic data representation based on a large annotated dataset, before adapting the representation to new…
Radioactive data: tracing through training
Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid +1
We want to detect whether a particular image dataset has been used to train a model. We propose a new technique, \emph{radioactive data}, that makes imperceptible changes to this d…
Optimized Generic Feature Learning for Few-shot Classification across Domains
Tonmoy Saikia, Thomas Brox, Cordelia Schmid
To learn models or features that generalize across tasks and domains is one of the grand goals of machine learning. In this paper, we propose to use cross-domain, cross-task data a…