65 citations · 476 across the 71 of their papers we have counts for
83 papers
An Empirical Study on What Matters for Viewpoint-Generalizable Policies in Visual Imitation Learning
Mino Nakura, Sriram Krishna, Yufei Wang +3
Visual imitation learning is a promising approach to training robot manipulation policies capable of completing a wide variety of tasks. However, policies today remain brittle to v…
Point4D: Long-range 4D Motion Reconstruction
Minsik Jeon, Jay Karhade, Deva Ramanan +1
We introduce Point4D, a feed-forward model for 4D reconstruction of long-range video sequences. Point4D is able to reliably infer dense per-point 3D trajectories across multi-hundr…
: Reactive Real-time Flow Policies
Sungjae Park, Shubham Tulsiani
Generalist manipulation policies increasingly take the form of action-chunking flow policies built on large pretrained backbones. Such chunks run open-loop, so the policy cannot re…
Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild
Yehonathan Litman, Xiaoxuan Ma, Manan Shah +4
Reconstructing dynamic non-rigid objects from monocular video requires integrating visual cues from direct observations with data-driven priors over geometry and appearance. Prior…
GHOST: Hierarchical Sub-Goal Policies for Generalizing Robot Manipulation
Sriram Krishna, Ben Eisner, Haotian Zhan +5
We present GHOST, a framework for learning visuomotor manipulation policies that generalize beyond the training distribution. GHOST factorizes control into (i) a high-level policy…
EditCtrl: Disentangled Local and Global Control for Real-Time Generative Video Editing
Yehonathan Litman, Shikun Liu, Dario Seyb +5
High-fidelity generative video editing has seen significant quality improvements by leveraging pre-trained video foundation models. However, their computational cost is a major bot…
Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning
Zhongxiao Cong, Qitao Zhao, Minsik Jeon +1
Current feed-forward 3D/4D reconstruction systems rely on dense geometry and pose supervision -- expensive to obtain at scale and particularly scarce for dynamic real-world scenes.…
Dex4D: Task-Agnostic Point Track Policy for Zero-Shot Dexterous Manipulation
Yuxuan Kuang, Sungjae Park, Katerina Fragkiadaki +1
Learning generalist policies capable of accomplishing a wide range of everyday tasks remains an open challenge in dexterous manipulation. In particular, collecting large-scale mani…
RayRoPE: Projective Ray Positional Encoding for Multi-view Attention
Yu Wu, Minsik Jeon, Jen-Hao Rick Chang +2
We study positional encodings for multi-view transformers that process tokens from a set of posed input images, and seek a mechanism that encodes patches uniquely, allows SE(3)-inv…
CRISP: Contact-Guided Real2Sim from Monocular Video with Planar Scene Primitives
Zihan Wang, Jiashun Wang, Jeff Tan +4
We introduce CRISP, a method that recovers simulatable human motion and scene geometry from monocular video. Prior work on joint human-scene reconstruction relies on data-driven pr…
E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training
Qitao Zhao, Hao Tan, Qianqian Wang +5
Self-supervised pre-training has driven rapid progress in foundation models for language, 2D images, and video, yet remains largely unexplored for learning 3D-aware representations…
Temporal Score Rescaling for Temperature Sampling in Diffusion and Flow Models
Yanbo Xu, Yu Wu, Sungjae Park +2
We present a mechanism to steer the sampling diversity of denoising diffusion and flow matching models, allowing users to sample from a sharper or broader distribution than the tra…
LightSwitch: Multi-view Relighting with Material-guided Diffusion
Yehonathan Litman, Fernando De la Torre, Shubham Tulsiani
Recent approaches for 3D relighting have shown promise in integrating 2D image relighting generative priors to alter the appearance of a 3D representation while preserving the unde…
DemoDiffusion: One-Shot Human Imitation using pre-trained Diffusion Policy
Sungjae Park, Homanga Bharadhwaj, Shubham Tulsiani
We propose DemoDiffusion, a simple method for enabling robots to perform manipulation tasks by imitating a single human demonstration, without requiring task-specific training or p…
UniPhy: Learning a Unified Constitutive Model for Inverse Physics Simulation
Himangi Mittal, Peiye Zhuang, Hsin-Ying Lee +1
We propose UniPhy, a common latent-conditioned neural constitutive model that can encode the physical properties of diverse materials. At inference UniPhy allows `inverse simulatio…
DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion
Qitao Zhao, Amy Lin, Jeff Tan +3
Current Structure-from-Motion (SfM) methods typically follow a two-stage pipeline, combining learned or geometric pairwise reasoning with a subsequent global optimization step. In…
Web2Grasp: Learning Functional Grasps from Web Images of Hand-Object Interactions
Hongyi Chen, Yunchao Yao, Yufei Ye +8
Functional grasping is essential for enabling dexterous multi-finger robot hands to manipulate objects effectively. Prior work largely focuses on power grasps, which only involve h…
AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis
Khiem Vuong, Anurag Ghosh, Deva Ramanan +2
We explore the task of geometric reconstruction of images captured from a mixture of ground and aerial views. Current state-of-the-art learning-based approaches fail to handle the…
Accenture-NVS1: A Novel View Synthesis Dataset
Thomas Sugg, Kyle O'Brien, Lekh Poudel +6
This paper introduces ACC-NVS1, a specialized dataset designed for research on Novel View Synthesis specifically for airborne and ground imagery. Data for ACC-NVS1 was collected in…
Predicting 4D Hand Trajectory from Monocular Videos
Yufei Ye, Yao Feng, Omid Taheri +3
We present HaPTIC, an approach that infers coherent 4D hand trajectories from monocular videos. Current video-based hand pose reconstruction methods primarily focus on improving fr…
Turbo3D: Ultra-fast Text-to-3D Generation
Hanzhe Hu, Tianwei Yin, Fujun Luan +6
We present Turbo3D, an ultra-fast text-to-3D system capable of generating high-quality Gaussian splatting assets in under one second. Turbo3D employs a rapid 4-step, 4-view diffusi…
Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis
Qitao Zhao, Shubham Tulsiani
Inferring the 3D structure underlying a set of multi-view images typically requires solving two co-dependent tasks -- accurate 3D reconstruction requires precise camera poses, and…
SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generation
Alexey Bokhovkin, Quan Meng, Shubham Tulsiani +1
We present SceneFactor, a diffusion-based approach for large-scale 3D scene generation that enables controllable generation and effortless editing. SceneFactor enables text-guided…
Diverse Score Distillation
Yanbo Xu, Jayanth Srinivasa, Gaowen Liu +1
Score distillation of 2D diffusion models has proven to be a powerful mechanism to guide 3D optimization, for example enabling text-based 3D generation or single-view reconstructio…