1 citations · 2 across the 13 of their papers we have counts for
14 papers
TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views
Skanda Koppula, Frano Rajic, Abdullah Faiz Ur Rahman +9
Multi-camera systems are increasingly practical for robotics, AR/VR, and autonomous driving because complementary views reduce depth ambiguity and preserve visibility under occlusi…
RSATalker: Realistic Socially-Aware Talking Head Generation for Multi-Turn Conversation
Peng Chen, Xiaobao Wei, Yi Yang +3
Talking head generation is increasingly important in virtual reality (VR), especially for social scenarios involving multi-turn conversation. Existing approaches face notable limit…
Recurrent Video Masked Autoencoders
Daniel Zoran, Nikhil Parthasarathy, Yi Yang +3
We present Recurrent Video Masked-Autoencoders (RVM): a novel approach to video representation learning that leverages recurrent computation to model the temporal structure of vide…
2nd Place Solution for CVPR2024 E2E Challenge: End-to-End Autonomous Driving Using Vision Language Model
Zilong Guo, Yi Luo, Long Sha +4
End-to-end autonomous driving has drawn tremendous attention recently. Many works focus on using modular deep neural networks to construct the end-to-end archi-tecture. However, wh…
FreeLong++: Training-Free Long Video Generation via Multi-band SpectralFusion
Yu Lu, Yi Yang
Recent advances in video generation models have enabled high-quality short video generation from text prompts. However, extending these models to longer videos remains a significan…
CHRIS: Clothed Human Reconstruction with Side View Consistency
Dong Liu, Yifan Yang, Zixiong Huang +2
Creating a realistic clothed human from a single-view RGB image is crucial for applications like mixed reality and filmmaking. Despite some progress in recent years, mainstream met…
TAPNext: Tracking Any Point (TAP) as Next Token Prediction
Artem Zholus, Carl Doersch, Yi Yang +7
Tracking Any Point (TAP) in a video is a challenging computer vision problem with many demonstrated applications in robotics, video editing, and 3D reconstruction. Existing methods…
From Image to Video: An Empirical Study of Diffusion Representations
Pedro Vélez, Luisa F. Polanía, Yi Yang +4
Diffusion models have revolutionized generative modeling, enabling unprecedented realism in image and video synthesis. This success has sparked interest in leveraging their represe…
Scaling 4D Representations
João Carreira, Dilara Gokay, Michael King +32
Scaling has not yet been convincingly demonstrated for pure self-supervised learning from video. However, prior work has focused evaluations on semantic-related tasks $\unicode{x20…
Moving Off-the-Grid: Scene-Grounded Video Representations
Sjoerd van Steenkiste, Daniel Zoran, Yi Yang +13
Current vision models typically maintain a fixed correspondence between their representation structure and image space. Each layer comprises a set of tokens arranged "on-the-grid,"…
Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Images
Ci Li, Yi Yang, Zehang Weng +3
In recent years, 3D parametric animal models have been developed to aid in estimating 3D shape and pose from images and video. While progress has been made for humans, it's more ch…
TAPVid-3D: A Benchmark for Tracking Any Point in 3D
Skanda Koppula, Ignacio Rocco, Yi Yang +5
We introduce a new benchmark, TAPVid-3D, for evaluating the task of long-range Tracking Any Point in 3D (TAP-3D). While point tracking in two dimensions (TAP) has many benchmarks m…
BootsTAP: Bootstrapped Training for Tracking-Any-Point
Carl Doersch, Pauline Luc, Yi Yang +8
To endow models with greater understanding of physics and motion, it is useful to enable them to perceive how solid surfaces move and deform in real scenes. This can be formalized…
Learning from One Continuous Video Stream
João Carreira, Michael King, Viorica Pătrăucean +9
We introduce a framework for online learning from a single continuous video stream -- the way people and animals learn, without mini-batches, data augmentation or shuffling. This p…