75 citations · 262 across the 24 of their papers we have counts for
13 papers · 1 filter
Replay: Multi-modal Multi-view Acted Videos for Casual Holography
Roman Shapovalov, Yanir Kleiman, Ignacio Rocco +6
We introduce Replay, a collection of multi-view, multi-modal videos of humans interacting socially. Each scene is filmed in high production quality, from different viewpoints with…
CLIP-Count: Towards Text-Guided Zero-Shot Object Counting
Ruixiang Jiang, Lingbo Liu, Changwen Chen
Recent advances in visual-language models have shown remarkable zero-shot text-image matching ability that is transferable to downstream tasks such as object detection and segmenta…
Novel-View Acoustic Synthesis
Changan Chen, Alexander Richard, Roman Shapovalov +4
We introduce the novel-view acoustic synthesis (NVAS) task: given the sight and sound observed at a source viewpoint, can we synthesize the sound of that scene from an unseen targe…
Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Training
Junfan Lin, Jianlong Chang, Lingbo Liu +4
Text-to-motion generation is an emerging and challenging problem, which aims to synthesize motion with the same semantics as the input text. However, due to the lack of diverse lab…
Retrospectives on the Embodied AI Workshop
Matt Deitke, Dhruv Batra, Yonatan Bisk +36
We present a retrospective on the state of Embodied AI research. Our analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR. These challenges are grouped i…
Towards a Unified View on Visual Parameter-Efficient Transfer Learning
Bruce X. B. Yu, Jianlong Chang, Lingbo Liu +2
Parameter efficient transfer learning (PETL) aims at making good use of the representation knowledge in the pre-trained large models by fine-tuning a small number of parameters. Re…
Taking an Emotional Look at Video Paragraph Captioning
Qinyu Li, Tengpeng Li, Hanli Wang +1
Translating visual data into natural language is essential for machines to understand the world and interact with humans. In this work, a comprehensive study is conducted on video…
Knowledge-enriched Attention Network with Group-wise Semantic for Visual Storytelling
Tengpeng Li, Hanli Wang, Bin He +1
As a technically challenging topic, visual storytelling aims at generating an imaginary and coherent story with narrative multi-sentences from a group of relevant images. Existing…
Visual Acoustic Matching
Changan Chen, Ruohan Gao, Paul Calamia +1
We introduce the visual acoustic matching task, in which an audio clip is transformed to sound like it was recorded in a target environment. Given an image of the target environmen…
Semantic Audio-Visual Navigation
Changan Chen, Ziad Al-Halah, Kristen Grauman
Recent work on audio-visual navigation assumes a constantly-sounding target and restricts the role of audio to signaling the target's position. We introduce semantic audio-visual n…
Learning to Set Waypoints for Audio-Visual Navigation
Changan Chen, Sagnik Majumder, Ziad Al-Halah +3
In audio-visual navigation, an agent intelligently travels through a complex, unmapped 3D environment using both sights and sounds to find a sound source (e.g., a phone ringing in…
VisualEchoes: Spatial Image Representation Learning through Echolocation
Ruohan Gao, Changan Chen, Ziad Al-Halah +2
Several animal species (e.g., bats, dolphins, and whales) and even visually impaired humans have the remarkable ability to perform echolocation: a biological sonar used to perceive…
SoundSpaces: Audio-Visual Navigation in 3D Environments
Changan Chen, Unnat Jain, Carl Schissler +5
Moving around in the world is naturally a multisensory experience, but today's embodied agents are deaf---restricted to solely their visual perception of the environment. We introd…