26 citations · 29 across the 26 of their papers we have counts for
26 papers
ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
Gehan Zheng, Matthew Johnson-Roberson, Weiming Zhi
Contact-rich manipulation failures are often detected only after the robot has committed to contact. This is especially limiting in wrist-camera setups: close gripper--object views…
One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models
Xiaohao Xu, Feng Xue, Xiang Li +5
A faithful 3D world representation should account for layered geometry, where a single camera ray may contain multiple visible and geometrically valid surfaces. Monocular depth est…
Rewind-IL: Online Failure Detection and State Respawning for Imitation Learning
Gehan Zheng, Sanjay Seenivasan, Matthew Johnson-Roberson +1
Imitation learning has enabled robots to acquire complex visuomotor manipulation skills from demonstrations, but deployment failures remain a major obstacle, especially for long-ho…
Cross-Modal Instructions for Robot Motion Generation
William Barron, Xiaoxiang Dong, Matthew Johnson-Roberson +1
Teaching robots novel behaviors typically requires motion demonstrations via teleoperation or kinaesthetic teaching, that is, physically guiding the robot. While recent work has ex…
Joint Flow Trajectory Optimization For Feasible Robot Motion Generation from Video Demonstrations
Xiaoxiang Dong, Matthew Johnson-Roberson, Weiming Zhi
Learning from human video demonstrations offers a scalable alternative to teleoperation or kinesthetic teaching, but poses challenges for robot manipulators due to embodiment diffe…
Efficient Construction of Implicit Surface Models From a Single Image for Motion Generation
Wei-Teng Chu, Tianyi Zhang, Matthew Johnson-Roberson +1
Implicit representations have been widely applied in robotics for obstacle avoidance and path planning. In this paper, we explore the problem of constructing an implicit distance r…
Bi-Manual Joint Camera Calibration and Scene Representation
Haozhan Tang, Tianyi Zhang, Matthew Johnson-Roberson +1
Robot manipulation, especially bimanual manipulation, often requires setting up multiple cameras on multiple robot manipulators. Before robot manipulators can generate motion or ev…
From Single Images to Motion Policies via Video-Generation Environment Representations
Weiming Zhi, Ziyong Ma, Tianyi Zhang +1
Autonomous robots typically need to construct representations of their surroundings and adapt their motions to the geometry of their environment. Here, we tackle the problem of con…
GraphSeg: Segmented 3D Representations via Graph Edge Addition and Contraction
Haozhan Tang, Tianyi Zhang, Oliver Kroemer +2
Robots operating in unstructured environments often require accurate and consistent object-level representations. This typically requires segmenting individual objects from the rob…
Infinite Leagues Under the Sea: Photorealistic 3D Underwater Terrain Generation by Latent Fractal Diffusion Models
Tianyi Zhang, Weiming Zhi, Joshua Mangelson +1
This paper tackles the problem of generating representations of underwater 3D terrain. Off-the-shelf generative models, trained on Internet-scale data but not on specialized underw…
Towards Ambiguity-Free Spatial Foundation Model: Rethinking and Decoupling Depth Ambiguity
Xiaohao Xu, Feng Xue, Xiang Li +5
Depth ambiguity is a fundamental challenge in spatial scene understanding, especially in transparent scenes where single-depth estimates fail to capture full 3D structure. Existing…
DOSE3 : Diffusion-based Out-of-distribution detection on SE(3) trajectories
Hongzhe Cheng, Tianyou Zheng, Tianyi Zhang +2
Out-of-Distribution(OOD) detection, a fundamental machine learning task aimed at identifying abnormal samples, traditionally requires model retraining for different inlier distribu…
Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video
Xiaohao Xu, Tianyi Zhang, Shibo Zhao +8
We aim to redefine robust ego-motion estimation and photorealistic 3D reconstruction by addressing a critical limitation: the reliance on noise-free data in existing models. While…
Robust Bayesian Scene Reconstruction with Retrieval-Augmented Priors for Precise Grasping and Planning
Herbert Wright, Weiming Zhi, Martin Matak +2
Constructing 3D representations of object geometry is critical for many robotics tasks, particularly manipulation problems. These representations must be built from potentially noi…
SplaTraj: Camera Trajectory Generation with Semantic Gaussian Splatting
Xinyi Liu, Tianyi Zhang, Matthew Johnson-Roberson +1
Many recent developments for robots to represent environments have focused on photorealistic reconstructions. This paper particularly focuses on generating sequences of images from…
PhotoReg: Photometrically Registering 3D Gaussian Splatting Models
Ziwen Yuan, Tianyi Zhang, Matthew Johnson-Roberson +1
Building accurate representations of the environment is critical for intelligent robots to make decisions during deployment. Advances in photorealistic environment models have enab…
Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation
Quanting Xie, So Yeon Min, Pengliang Ji +7
There is no limit to how much a robot might explore and learn, but all of that knowledge needs to be searchable and actionable. Within language research, retrieval augmented genera…
RecGS: Removing Water Caustic with Recurrent Gaussian Splatting
Tianyi Zhang, Weiming Zhi, Kaining Huang +3
Water caustics are commonly observed in seafloor imaging data from shallow-water areas. Traditional methods that remove caustic patterns from images often rely on 2D filtering or p…
3D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objects
Weiming Zhi, Haozhan Tang, Tianyi Zhang +1
Humans have the remarkable ability to use held objects as tools to interact with their environment. For this to occur, humans internally estimate how hand movements affect the obje…
From Perfect to Noisy World Simulation: Customizable Embodied Multi-modal Perturbations for SLAM Robustness Benchmarking
Xiaohao Xu, Tianyi Zhang, Sibo Wang +6
Embodied agents require robust navigation systems to operate in unstructured environments, making the robustness of Simultaneous Localization and Mapping (SLAM) models critical to…
Learning Shared RGB-D Fields: Unified Self-supervised Pre-training for Label-efficient LiDAR-Camera 3D Perception
Xiaohao Xu, Ye Li, Tianyi Zhang +3
Constructing large-scale labeled datasets for multi-modal perception model training in autonomous driving presents significant challenges. This has motivated the development of sel…
Unifying Scene Representation and Hand-Eye Calibration with 3D Foundation Models
Weiming Zhi, Haozhan Tang, Tianyi Zhang +1
Representing the environment is a central challenge in robotics, and is essential for effective decision-making. Traditionally, before capturing images with a manipulator-mounted c…
Teaching Robots Where To Go And How To Act With Human Sketches via Spatial Diagrammatic Instructions
Qilin Sun, Weiming Zhi, Tianyi Zhang +1
This paper introduces Spatial Diagrammatic Instructions (SDIs), an approach for human operators to specify objectives and constraints that are related to spatial regions in the wor…
DarkGS: Learning Neural Illumination and 3D Gaussians Relighting for Robotic Exploration in the Dark
Tianyi Zhang, Kaining Huang, Weiming Zhi +1
Humans have the remarkable ability to construct consistent mental models of an environment, even under limited or varying levels of illumination. We wish to endow robots with this…