197 citations · 349 across the 52 of their papers we have counts for
61 papers
OmniShape: Zero-Shot Multi-Hypothesis Shape and Pose Estimation in the Real World
Katherine Liu, Sergey Zakharov, Dian Chen +4
We would like to estimate the pose and full shape of an object from a single observation, without assuming known 3D model or category. In this work, we propose OmniShape, the first…
Video Generators are Robot Policies
Junbang Liang, Pavel Tokmakov, Ruoshi Liu +4
Despite tremendous progress in dexterous manipulation, current visuomotor policies remain fundamentally limited by two challenges: they struggle to generalize under perceptual or b…
AllTracker: Efficient Dense Point Tracking at High Resolution
Adam W. Harley, Yang You, Xinglong Sun +11
We introduce AllTracker: a model that estimates long-range point tracks by way of estimating the flow field between a query frame and every other frame of a video. Unlike existing…
A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation
TRI LBM Team, Jose Barreiros, Andrew Beaulieu +79
Robot manipulation has seen tremendous progress in recent years, with imitation learning policies enabling successful performance of dexterous and hard-to-model tasks. Concurrently…
SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting
Shengjie Lin, Jiading Fang, Muhammad Zubair Irshad +4
Reconstructing articulated objects prevalent in daily environments is crucial for applications in augmented/virtual reality and robotics. However, existing methods face scalability…
Understanding Complexity in VideoQA via Visual Program Generation
Cristobal Eyzaguirre, Igor Vasiljevic, Achal Dave +5
We propose a data-driven approach to analyzing query complexity in Video Question Answering (VideoQA). Previous efforts in benchmark design have relied on human expertise to design…
GTR: Gaussian Splatting Tracking and Reconstruction of Unknown Objects Based on Appearance and Geometric Complexity
Takuya Ikeda, Sergey Zakharov, Muhammad Zubair Irshad +8
We present a novel method for 6-DoF object tracking and high-quality 3D reconstruction from monocular RGBD video. Existing methods, while achieving impressive results, often strugg…
Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware
Justin Yu, Letian Fu, Huang Huang +5
Scaling robot learning requires vast and diverse datasets. Yet the prevailing data collection paradigm-human teleoperation-remains costly and constrained by manual effort and physi…
ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Grasping
Shun Iwase, Zubair Irshad, Katherine Liu +8
Robotic grasping is a cornerstone capability of embodied systems. Many methods directly output grasps from partial information without modeling the geometry of the scene, leading t…
Is Your Imitation Learning Policy Better than Mine? Policy Comparison with Near-Optimal Stopping
David Snyder, Asher James Hancock, Apurva Badithela +6
Imitation learning has enabled robots to perform complex, long-horizon tasks in challenging dexterous manipulation settings. As new methods are developed, they must be rigorously e…
Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies
Chen Xu, Tony Khuong Nguyen, Emma Dixon +7
Recent years have witnessed impressive robotic manipulation systems driven by advances in imitation learning and generative modeling, such as diffusion- and flow-based approaches.…
Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion
Vitor Guizilini, Muhammad Zubair Irshad, Dian Chen +2
Current methods for 3D scene reconstruction from sparse posed images employ intermediate 3D representations such as neural fields, voxel grids, or 3D Gaussians, to achieve multi-vi…
Espresso: High Compression For Rich Extraction From Videos for Your Vision-Language Model
Keunwoo Peter Yu, Achal Dave, Rares Ambrus +1
Recent advances in vision-language models (VLMs) have shown great promise in connecting images and text, but extending these models to long videos remains challenging due to the ra…
Equivariant Ray Embeddings for Implicit Multi-View Depth Estimation
Yinshuang Xu, Dian Chen, Katherine Liu +4
Incorporating inductive bias by embedding geometric entities (such as rays) as input has proven successful in multi-view learning. However, the methods adopting this technique typi…
Neural Fields in Robotics: A Survey
Muhammad Zubair Irshad, Mauro Comi, Yen-Chen Lin +5
Neural Fields have emerged as a transformative approach for 3D scene representation in computer vision and robotics, enabling accurate inference of geometry, 3D semantics, and dyna…
GRIN: Zero-Shot Metric Depth with Pixel-Level Diffusion
Vitor Guizilini, Pavel Tokmakov, Achal Dave +1
3D reconstruction from a single image is a long-standing problem in computer vision. Learning-based methods address its inherent scale ambiguity by leveraging increasingly large la…
Incorporating dense metric depth into neural 3D representations for view synthesis and relighting
Arkadeep Narayan Chaudhury, Igor Vasiljevic, Sergey Zakharov +4
Synthesizing accurate geometry and photo-realistic appearance of small scenes is an active area of research with compelling use cases in gaming, virtual reality, robotic-manipulati…
ReFiNe: Recursive Field Networks for Cross-modal Multi-scene Representation
Sergey Zakharov, Katherine Liu, Adrien Gaidon +1
The common trade-offs of state-of-the-art methods for multi-shape representation (a single model "packing" multiple objects) involve trading modeling accuracy against memory and st…
Transcrib3D: 3D Referring Expression Resolution through Large Language Models
Jiading Fang, Xiangshan Tan, Shengjie Lin +6
If robots are to work effectively alongside people, they must be able to interpret natural language references to objects in their 3D environment. Understanding 3D referring expres…
NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fields
Muhammad Zubair Irshad, Sergey Zakharov, Vitor Guizilini +3
Neural fields excel in computer vision and robotics due to their ability to understand the 3D visual world such as inferring semantics, geometry, and dynamics. Given the capabiliti…
DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation
Takuya Ikeda, Sergey Zakharov, Tianyi Ko +5
This paper addresses the challenging problem of category-level pose estimation. Current state-of-the-art methods for this task face challenges when dealing with symmetric objects a…
Zero-Shot Multi-Object Scene Completion
Shun Iwase, Katherine Liu, Vitor Guizilini +4
We present a 3D scene completion method that recovers the complete geometry of multiple unseen objects in complex scenes from a single RGB-D image. Despite notable advancements in…
Understanding Video Transformers via Universal Concept Discovery
Matthew Kowal, Achal Dave, Rares Ambrus +3
This paper studies the problem of concept-based interpretability of transformer representations for videos. Concretely, we seek to explain the decision-making process of video tran…
FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects
Mayank Lunayach, Sergey Zakharov, Dian Chen +3
In this work, we address the challenging task of 3D object recognition without the reliance on real-world 3D labeled data. Our goal is to predict the 3D shape, size, and 6D pose of…