349 citations · 2.5k across the 74 of their papers we have counts for
13 papers · 1 filter
Look, Listen, and Act: Towards Audio-Visual Embodied Navigation
Chuang Gan, Yiwei Zhang, Jiajun Wu +2
A crucial ability of mobile intelligent agents is to integrate the evidence from multiple sensory inputs in an environment and to make a sequence of actions to reach their goals. I…
Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement
Chao Yang, Xiaojian Ma, Wenbing Huang +4
This paper studies Learning from Observations (LfO) for imitation learning with access to state-only demonstrations. In contrast to Learning from Demonstration (LfD) that involves…
Self-supervised Moving Vehicle Tracking with Stereo Sound
Chuang Gan, Hang Zhao, Peihao Chen +2
Humans are able to localize objects in the environment using both visual and auditory cues, integrating information from multiple modalities into a common reference frame. We intro…
TruNet: Short Videos Generation from Long Videos via Story-Preserving Truncation
Fan Yang, Xiao Liu, Dongliang He +5
In this work, we introduce a new problem, named as {\em story-preserving long video truncation}, that requires an algorithm to automatically truncate a long-duration video into mul…
CLEVRER: CoLlision Events for Video REpresentation and Reasoning
Kexin Yi, Chuang Gan, Yunzhu Li +4
The ability to reason about temporal and causal events from videos lies at the core of human intelligence. Most video reasoning benchmarks, however, focus on pattern recognition fr…
Training Kinetics in 15 Minutes: Large-scale Distributed Training on Videos
Ji Lin, Chuang Gan, Song Han
Deep video recognition is more computationally expensive than image recognition, especially on large-scale datasets like Kinetics [1]. Therefore, training scalability is essential…
Graph Convolutional Networks for Temporal Action Localization
Runhao Zeng, Wenbing Huang, Mingkui Tan +4
Most state-of-the-art action localization systems process each action proposal individually, without explicitly exploiting their relations during learning. However, the relations b…
Deep Concept-wise Temporal Convolutional Networks for Action Localization
Xin Li, Tianwei Lin, Xiao Liu +7
Existing action localization approaches adopt shallow temporal convolutional networks (\ie, TCN) on 1D feature map extracted from video frames. In this paper, we empirically find t…
Once-for-All: Train One Network and Specialize it for Efficient Deployment
Han Cai, Chuang Gan, Tianzhe Wang +2
We address the challenging problem of efficient inference across many devices and resource constraints, especially on edge devices. Conventional approaches either manually design o…
The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
Jiayuan Mao, Chuang Gan, Pushmeet Kohli +2
We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that learns visual concepts, words, and semantic parsing of sentences without explicit supervision on any of them; in…
Self-Supervised Audio-Visual Co-Segmentation
Andrew Rouditchenko, Hang Zhao, Chuang Gan +2
Segmenting objects in images and separating sound sources in audio are challenging tasks, in part because traditional approaches require large amounts of labeled data. In this pape…
The Sound of Motions
Hang Zhao, Chuang Gan, Wei-Chiu Ma +1
Sounds originate from object motions and vibrations of surrounding air. Inspired by the fact that humans is capable of interpreting sound sources from how objects move visually, we…
Interpreting Adversarial Examples by Activation Promotion and Suppression
Kaidi Xu, Sijia Liu, Gaoyuan Zhang +5
It is widely known that convolutional neural networks (CNNs) are vulnerable to adversarial examples: images with imperceptible perturbations crafted to fool classifiers. However, i…