49 citations · 59 across the 19 of their papers we have counts for
22 papers
GATOR: Generative and Agentic 3D Object Reconstruction From Casual Images
Qirui Wu, Stan Birchfield, Hesam Rabeti +2
Reconstructing complete, scene-aligned 3D objects from casual images requires integrating sparse, uncertain observations and inferring surfaces hidden by occlusions. We present GAT…
Hydra-0: Action Flow for Generalist World Modeling and Control
Hongyu Li, Bowen Wen, Xinghao Zhu +8
We introduce Hydra-0, a generalist world model conditioned on action flow, which represents robot actions as pixel motion. This shared visual interface enables generalist world mod…
MG-VQA: Manipulation Grounded Visual Question Answering with VLMs
Vineet Bhat, Siyi Chen, Mikaela Angelina Uy +6
Vision-language models (VLMs) have shown promising spatial reasoning capabilities from static visual inputs, where the evidence needed to answer a question is available in the prov…
Vesta: A Generalist Embodied Reasoning Model
Johan Bjorck, Zhiqi Li, Yunze Man +29
Robots operating in open-world environments must seamlessly integrate localization, spatial reasoning, navigation, and long-horizon planning. While specialist models excel at indiv…
See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents
Siyi Chen, Xiaoyan Zhang, Meng Wu +7
Language-model agents build internal representations of the information they observe and the reasoning they perform. Sharing these representations offers a way to communicate both…
VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation
Siyi Chen, Hugo Hadfield, Alex Zook +9
Open-vocabulary long-horizon manipulation requires robots to reason over flexible instructions and complex multi-object scenes while adaptively planning, executing, monitoring, and…
GraspGen-X: Cross-Embodiment 6-DOF Diffusion-based Grasping
Beining Han, Yu-Wei Chao, Erwin Coumans +5
We study cross-embodiment 6-DOF robot grasping. Unlike prior works, we require the model not only to generalize to novel objects / scenes but also to novel gripper morphologies and…
RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
Jenai Xuning Yang, Rishit Dagli, Alex Zook +5
The pursuit of general-purpose robotics has yielded impressive foundation models, yet simulation-based benchmarking remains a bottleneck due to rapid performance saturation and a l…
CARI4D: Category Agnostic 4D Reconstruction of Human-Object Interaction
Xianghui Xie, Bowen Wen, Yan Chang +5
Accurate capture of human-object interaction from ubiquitous sensors like RGB cameras is important for applications in human understanding, gaming, and robot learning. However, inf…
SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL
Siyi Chen, Mikaela Angelina Uy, Chan Hee Song +6
Vision Language Models (VLMs) demonstrate strong qualitative visual understanding, but struggle with metrically precise spatial reasoning required for embodied applications. The ag…
BOP-ASK: Object-Interaction Reasoning for Vision-Language Models
Vineet Bhat, Sungsu Kim, Valts Blukis +6
Vision Language Models (VLMs) have achieved impressive performance on spatial reasoning benchmarks, yet these evaluations mask critical weaknesses in understanding object interacti…
3D-Generalist: Self-Improving Vision-Language-Action Models for Crafting 3D Worlds
Fan-Yun Sun, Shengguang Wu, Christian Jacobsen +13
Despite large-scale pretraining endowing models with language and vision reasoning capabilities, improving their spatial reasoning capability remains challenging due to the lack of…
Robot Policy Evaluation for Sim-to-Real Transfer: A Benchmarking Perspective
Xuning Yang, Clemens Eppner, Jonathan Tremblay +3
Current vision-based robotics simulation benchmarks have significantly advanced robotic manipulation research. However, robotics is fundamentally a real-world problem, and evaluati…
GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Training
Adithyavairavan Murali, Balakumar Sundaralingam, Yu-Wei Chao +7
Grasping is a fundamental robot skill, yet despite significant research advancements, learning-based 6-DOF grasping approaches are still not turnkey and struggle to generalize acro…
OG-VLA: Orthographic Image Generation for 3D-Aware Vision-Language Action Model
Ishika Singh, Ankit Goyal, Stan Birchfield +3
We introduce OG-VLA, a novel architecture and learning framework that combines the generalization strengths of Vision Language Action models (VLAs) with the robustness of 3D-aware…
BOP Challenge 2024 on Model-Based and Model-Free 6D Object Pose Estimation
Van Nguyen Nguyen, Stephen Tyree, Andrew Guo +16
We present the evaluation methodology, datasets and results of the BOP Challenge 2024, the 6th in a series of public competitions organized to capture the state of the art in 6D ob…
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
Chan Hee Song, Valts Blukis, Jonathan Tremblay +3
Spatial understanding is a crucial capability that enables robots to perceive their surroundings, reason about their environment, and interact with it meaningfully. In modern robot…
Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models
Alexander Popov, Alperen Degirmenci, David Wehr +9
We propose the use of latent space generative world models to address the covariate shift problem in autonomous driving. A world model is a neural network capable of predicting an…
NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows
Zhenggang Tang, Zhongzheng Ren, Xiaoming Zhao +4
We present a method for automatically modifying a NeRF representation based on a single observation of a non-rigid transformed version of the original scene. Our method defines the…
Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective
Zhen Qin, Xuyang Shen, Dong Li +4
We present the Linear Complexity Sequence Model (LCSM), a comprehensive solution that unites various sequence modeling techniques with linear complexity, including linear attention…
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Bowen Wen, Wei Yang, Jan Kautz +1
We present FoundationPose, a unified foundation model for 6D object pose estimation and tracking, supporting both model-based and model-free setups. Our approach can be instantly a…
Riemannian Motion Policies
Nathan D. Ratliff, Jan Issac, Daniel Kappler +2
We introduce the Riemannian Motion Policy (RMP), a new mathematical object for modular motion generation. An RMP is a second-order dynamical system (acceleration field or motion po…