580 citations · 1.9k across the 92 of their papers we have counts for
83 papers · 1 filter
Surflo: Consistent 3D Surface Flow Model with Global State
Antoine Guédon, Shu Nakamura, Nicolas Dufour +3
Geometry is invariant to viewpoint, which makes any collection of images a redundant encoding of a single 3D state. Existing feed-forward reconstruction models fail to exploit this…
Self-Improving 4D Perception via Self-Distillation
Nan Huang, Pengcheng Yu, Weijia Zeng +4
Large-scale multi-view reconstruction models have made remarkable progress, but most existing approaches still rely on fully supervised training with ground-truth 3D/4D annotations…
Human-level 3D shape perception emerges from multi-view learning
Tyler Bonnen, Jitendra Malik, Angjoo Kanazawa
Humans can infer the three-dimensional structure of objects from two-dimensional visual inputs. Modeling this ability has been a longstanding goal for the science and engineering o…
Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models
David McAllister, Miika Aittala, Tero Karras +4
Reinforcement learning (RL) has become a standard technique for post-training diffusion-based image synthesis models, as it enables learning from reward signals to explicitly impro…
Visually Prompted Benchmarks Are Surprisingly Fragile
Haiwen Feng, Long Lian, Lisa Dunlap +6
A key challenge in evaluating VLMs is testing models' ability to analyze visual content independently from their textual priors. Recent benchmarks such as BLINK probe visual percep…
Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning
Shaofeng Yin, Jiaxin Ge, Zora Zhiruo Wang +6
Vision-as-inverse-graphics, the concept of reconstructing images into editable programs, remains challenging for Vision-Language Models (VLMs), which inherently lack fine-grained s…
Diffusion Forcing for Multi-Agent Interaction Sequence Modeling
Vongani H. Maluleke, Kie Horiuchi, Lea Wilken +3
Understanding and generating multi-person interactions is a fundamental challenge with broad implications for robotics and social computing. While humans naturally coordinate in gr…
Viser: Imperative, Web-based 3D Visualization in Python
Brent Yi, Chung Min Kim, Justin Kerr +8
We present Viser, a 3D visualization library for computer vision and robotics. Viser aims to bring easy and extensible 3D visualization to Python: we provide a comprehensive set of…
Cameras as Relative Positional Encoding
Ruilong Li, Brent Yi, Junchen Liu +3
Transformers are increasingly prevalent for multi-view computer vision tasks, where geometric relationships between viewpoints are critical for 3D perception. To leverage these rel…
Predict-Optimize-Distill: A Self-Improving Cycle for 4D Object Understanding
Mingxuan Wu, Huang Huang, Justin Kerr +4
Humans can resort to long-form inspection to build intuition on predicting the 3D configurations of unseen objects. The more we observe the object motion, the better we get at pred…
St4RTrack: Simultaneous 4D Reconstruction and Tracking in the World
Haiwen Feng, Junyi Zhang, Qianqian Wang +5
Dynamic 3D reconstruction and point tracking in videos are typically treated as separate tasks, despite their deep connection. We propose St4RTrack, a feed-forward framework that s…
Segment Any Motion in Videos
Nan Huang, Wenzhao Zheng, Chenfeng Xu +4
Moving object segmentation is a crucial task for achieving a high-level understanding of visual scenes and has numerous downstream applications. Humans can effortlessly segment mov…
Fillerbuster: Unified Generative Scene Completion Model for Casual Captures
Ethan Weber, Norman Müller, Yash Kant +4
We present Fillerbuster, a unified model that completes unknown regions of a 3D scene with a multi-view latent diffusion transformer. Casual captures are often sparse and miss surr…
Continuous 3D Perception Model with Persistent State
Qianqian Wang, Yifei Zhang, Aleksander Holynski +2
We present a unified framework capable of solving a broad range of 3D tasks. Our approach features a stateful recurrent model that continuously updates its state representation wit…
Decentralized Diffusion Models
David McAllister, Matthew Tancik, Jiaming Song +1
Large-scale AI model training divides work across thousands of GPUs, then synchronizes gradients across them at each step. This incurs a significant network burden that only centra…
MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videos
Zhengqi Li, Richard Tucker, Forrester Cole +6
We present a system that allows for accurate, fast, and robust estimation of camera parameters and depth maps from casual monocular videos of dynamic scenes. Most conventional stru…
Reconstructing People, Places, and Cameras
Lea Müller, Hongsuk Choi, Anthony Zhang +3
We present "Humans and Structure from Motion" (HSfM), a method for jointly reconstructing multiple human meshes, scene point clouds, and camera parameters in a metric world coordin…
SOAR: Self-Occluded Avatar Recovery from a Single Video In the Wild
Zhuoyang Pan, Angjoo Kanazawa, Hang Gao
Self-occlusion is common when capturing people in the wild, where the performer do not follow predefined motion scripts. This challenges existing monocular human reconstruction sys…
Agent-to-Sim: Learning Interactive Behavior Models from Casual Longitudinal Videos
Gengshan Yang, Andrea Bajcsy, Shunsuke Saito +1
We present Agent-to-Sim (ATS), a framework for learning interactive behavior models of 3D agents from casual longitudinal video collections. Different from prior works that rely on…
Estimating Body and Hand Motion in an Ego-sensed World
Brent Yi, Vickie Ye, Maya Zheng +6
We present EgoAllo, a system for human motion estimation from a head-mounted device. Using only egocentric SLAM poses and images, EgoAllo guides sampling from a conditional diffusi…
gsplat: An Open-Source Library for Gaussian Splatting
Vickie Ye, Ruilong Li, Justin Kerr +8
gsplat is an open-source library designed for training and developing Gaussian Splatting methods. It features a front-end with Python bindings compatible with the PyTorch library a…
Synergy and Synchrony in Couple Dances
Vongani Maluleke, Lea Müller, Jathushan Rajasegaran +4
This paper asks to what extent social interaction influences one's behavior. We study this in the setting of two dancers dancing as a couple. We first consider a baseline in which…
Shape of Motion: 4D Reconstruction from a Single Video
Qianqian Wang, Vickie Ye, Hang Gao +4
Monocular dynamic reconstruction is a challenging and long-standing vision problem due to the highly ill-posed nature of the task. Existing approaches depend on templates, are effe…
Splatfacto-W: A Nerfstudio Implementation of Gaussian Splatting for Unconstrained Photo Collections
Congrong Xu, Justin Kerr, Angjoo Kanazawa
Novel view synthesis from unconstrained in-the-wild image collections remains a significant yet challenging task due to photometric variations and transient occluders that complica…