2.9k citations · 4.6k across the 16 of their papers we have counts for
16 papers
Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding
Arsha Nagrani, Jasper Uijilings, Shyamal Buch +6
Video reasoning models are a core component of egocentric and embodied agents. However, standard benchmarks for assessing models provide only evaluation of the output (e.g. the ans…
IC3: Image Captioning by Committee Consensus
David M. Chan, Austin Myers, Sudheendra Vijayanarasimhan +2
If you ask a human to describe an image, they might do so in a thousand different ways. Traditionally, image captioning models are trained to generate a single "best" (most like a…
Open-Vocabulary Temporal Action Detection with Off-the-Shelf Image-Text Features
Vivek Rathod, Bryan Seybold, Sudheendra Vijayanarasimhan +4
Detecting actions in untrimmed videos should not be limited to a small, closed set of classes. We present a simple, yet effective strategy for open-vocabulary temporal action detec…
Distribution Aware Metrics for Conditional Natural Language Generation
David M Chan, Yiming Ni, David A Ross +3
Traditional automated metrics for evaluating conditional natural language generation use pairwise comparisons between a single generated text and the best-matching gold-standard gr…
What's in a Caption? Dataset-Specific Linguistic Diversity and Its Effect on Visual Description Models and Metrics
David M. Chan, Austin Myers, Sudheendra Vijayanarasimhan +3
While there have been significant gains in the field of automated video description, the generalization performance of automated description models to novel domains remains a major…
Active Learning for Video Description With Cluster-Regularized Ensemble Ranking
David M. Chan, Sudheendra Vijayanarasimhan, David A. Ross +1
Automatic video captioning aims to train models to generate text descriptions for all segments in a video, however, the most effective approaches require large amounts of manual an…
Rethinking the Faster R-CNN Architecture for Temporal Action Localization
Yu-Wei Chao, Sudheendra Vijayanarasimhan, Bryan Seybold +3
We propose TAL-Net, an improved approach to temporal action localization in video that is inspired by the Faster R-CNN object detection framework. TAL-Net addresses three key short…
End-to-End Learning of Semantic Grasping
Eric Jang, Sudheendra Vijayanarasimhan, Peter Pastor +2
We consider the task of semantic robotic grasping, in which a robot picks up an object of a user-specified class using only monocular images. Inspired by the two-stream hypothesis…
The Kinetics Human Action Video Dataset
Will Kay, Joao Carreira, Karen Simonyan +9
We describe the DeepMind Kinetics human action video dataset. The dataset contains 400 human action classes, with at least 400 video clips for each action. Each clip lasts around 1…
Motion Prediction Under Multimodality with Conditional Stochastic Networks
Katerina Fragkiadaki, Jonathan Huang, Alex Alemi +3
Given a visual history, multiple future outcomes for a video scene are equally probable, in other words, the distribution of future outcomes has multiple modes. Multimodality is no…
AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions
Chunhui Gu, Chen Sun, David A. Ross +9
This paper introduces a video dataset of spatio-temporally localized Atomic Visual Actions (AVA). The AVA dataset densely annotates 80 atomic visual actions in 430 15-minute video…
SfM-Net: Learning of Structure and Motion from Video
Sudheendra Vijayanarasimhan, Susanna Ricco, Cordelia Schmid +2
We propose SfM-Net, a geometry-aware neural network for motion estimation in videos that decomposes frame-to-frame pixel motion in terms of scene and object depth, camera motion an…
YouTube-8M: A Large-Scale Video Classification Benchmark
Sami Abu-El-Haija, Nisarg Kothari, Joonseok Lee +4
Many recent advancements in Computer Vision are attributed to large datasets. Open-source software packages for Machine Learning and inexpensive commodity hardware have reduced the…
Efficient Large Scale Video Classification
Balakrishnan Varadarajan, George Toderici, Sudheendra Vijayanarasimhan +1
Video classification has advanced tremendously over the recent years. A large part of the improvements in video classification had to do with the work done by the image classificat…
Beyond Short Snippets: Deep Networks for Video Classification
Joe Yue-Hei Ng, Matthew Hausknecht, Sudheendra Vijayanarasimhan +3
Convolutional neural networks (CNNs) have been extensively applied for image recognition problems giving state-of-the-art results on recognition, detection, segmentation and retrie…
Deep Networks With Large Output Spaces
Sudheendra Vijayanarasimhan, Jonathon Shlens, Rajat Monga +1
Deep neural networks have been extremely successful at various image, speech, video recognition tasks because of their ability to model deep structures within the data. However, th…