7 papers
Whareformer: Learning to Track What is Where in Long Egocentric Videos
Jacob Chalk, Saptarshi Sinha, Dima Damen +2
The recently established 'Out of Sight, Not out of Mind' (OSNOM) task for egocentric videos focuses on tracking objects that are moved by the camera wearer, online, maintaining kno…
Syn4D: A Multiview Synthetic 4D Dataset
Zeren Jiang, Yushi Lan, Yihang Luo +8
Dense 3D reconstruction and tracking of dynamic scenes from monocular video remains an important open challenge in computer vision. Progress in this area has been constrained by th…
Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Video
Zeren Jiang, Chuanxia Zheng, Iro Laina +2
We propose Mesh4D, a feed-forward model for monocular 4D mesh reconstruction. Given a monocular video of a dynamic object, our model reconstructs the object's complete 3D shape and…
Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction
Zeren Jiang, Chuanxia Zheng, Iro Laina +2
We introduce Geo4D, a method to repurpose video diffusion models for monocular 3D reconstruction of dynamic scenes. By leveraging the strong dynamic priors captured by large-scale…
Layered Motion Fusion: Lifting Motion Segmentation to 3D in Egocentric Videos
Vadim Tschernezki, Diane Larlus, Iro Laina +1
Computer vision is largely based on 2D techniques, with 3D vision still relegated to a relatively narrow subset of applications. However, by building on recent advances in 3D model…
What could go wrong? Discovering and describing failure modes in computer vision
Gabriela Csurka, Tyler L. Hayes, Diane Larlus +1
Deep learning models are effective, yet brittle. Even carefully trained, their behavior tends to be hard to predict when confronted with out-of-distribution samples. In this work,…