10 papers
EC: Video Generation with 3D Environmental Memory and Ego-Exo Human Pose Control
Qiao Gu, Lingni Ma, Adam W Harley +3
Controllable and physically grounded egocentric video generation is essential for embodied agents to reason about how their own and others' actions manifest and change the world. C…
LAMP: Localization Aware Multi-camera People Tracking in Metric 3D World
Nan Yang, Julian Straub, Fan Zhang +3
Tracking 3D human motion from egocentric multi-camera headset is challenged by severe egomotion, partial visibility or occlusions and lack of training data. Existing methods design…
Boxer: Robust Lifting of Open-World 2D Bounding Boxes to 3D
Daniel DeTone, Tianwei Shen, Fan Zhang +4
Detecting and localizing objects in space is a fundamental computer vision problem. While much progress has been made to solve 2D object detection, 3D object localization is much l…
NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Data
Daniel DeTone, Federica Bogo, Eric-Tuan Le +7
The Nymeria Dataset, released in 2024, is a large-scale collection of in-the-wild human activities captured with multiple egocentric wearable devices that are spatially localized a…
ShapeR: Robust Conditional 3D Shape Generation from Casual Captures
Yawar Siddiqui, Duncan Frost, Samir Aroudj +9
Recent advances in 3D shape generation have achieved impressive results, but most existing methods rely on clean, unoccluded, and well-segmented inputs. Such conditions are rarely…
DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos
Chieh Hubert Lin, Zhaoyang Lv, Songyin Wu +11
We introduce the Deformable Gaussian Splats Large Reconstruction Model (DGS-LRM), the first feed-forward method predicting deformable 3D Gaussian splats from a monocular posed vide…
Sonata: Self-Supervised Learning of Reliable Point Representations
Xiaoyang Wu, Daniel DeTone, Duncan Frost +7
In this paper, we question whether we have a reliable self-supervised point cloud model that can be used for diverse 3D tasks via simple linear probing, even with limited data and…
Human-in-the-Loop Local Corrections of 3D Scene Layouts via Infilling
Christopher Xie, Armen Avetisyan, Henry Howard-Jenkins +5
We present a novel human-in-the-loop approach to estimate 3D scene layout that uses human feedback from an egocentric standpoint. We study this approach through introduction of a n…
EFM3D: A Benchmark for Measuring Progress Towards 3D Egocentric Foundation Models
Julian Straub, Daniel DeTone, Tianwei Shen +3
The advent of wearable computers enables a new source of context for AI that is embedded in egocentric sensor data. This new egocentric data comes equipped with fine-grained 3D loc…
EgoLifter: Open-world 3D Segmentation for Egocentric Perception
Qiao Gu, Zhaoyang Lv, Duncan Frost +3
In this paper we present EgoLifter, a novel system that can automatically segment scenes captured from egocentric sensors into a complete decomposition of individual 3D objects. Th…