26 citations · 32 across the 24 of their papers we have counts for
24 papers
One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models
Xiaohao Xu, Feng Xue, Xiang Li +5
A faithful 3D world representation should account for layered geometry, where a single camera ray may contain multiple visible and geometrically valid surfaces. Monocular depth est…
Efficient Construction of Implicit Surface Models From a Single Image for Motion Generation
Wei-Teng Chu, Tianyi Zhang, Matthew Johnson-Roberson +1
Implicit representations have been widely applied in robotics for obstacle avoidance and path planning. In this paper, we explore the problem of constructing an implicit distance r…
Bi-Manual Joint Camera Calibration and Scene Representation
Haozhan Tang, Tianyi Zhang, Matthew Johnson-Roberson +1
Robot manipulation, especially bimanual manipulation, often requires setting up multiple cameras on multiple robot manipulators. Before robot manipulators can generate motion or ev…
From Single Images to Motion Policies via Video-Generation Environment Representations
Weiming Zhi, Ziyong Ma, Tianyi Zhang +1
Autonomous robots typically need to construct representations of their surroundings and adapt their motions to the geometry of their environment. Here, we tackle the problem of con…
GraphSeg: Segmented 3D Representations via Graph Edge Addition and Contraction
Haozhan Tang, Tianyi Zhang, Oliver Kroemer +2
Robots operating in unstructured environments often require accurate and consistent object-level representations. This typically requires segmenting individual objects from the rob…
Infinite Leagues Under the Sea: Photorealistic 3D Underwater Terrain Generation by Latent Fractal Diffusion Models
Tianyi Zhang, Weiming Zhi, Joshua Mangelson +1
This paper tackles the problem of generating representations of underwater 3D terrain. Off-the-shelf generative models, trained on Internet-scale data but not on specialized underw…
Towards Ambiguity-Free Spatial Foundation Model: Rethinking and Decoupling Depth Ambiguity
Xiaohao Xu, Feng Xue, Xiang Li +5
Depth ambiguity is a fundamental challenge in spatial scene understanding, especially in transparent scenes where single-depth estimates fail to capture full 3D structure. Existing…
DOSE3 : Diffusion-based Out-of-distribution detection on SE(3) trajectories
Hongzhe Cheng, Tianyou Zheng, Tianyi Zhang +2
Out-of-Distribution(OOD) detection, a fundamental machine learning task aimed at identifying abnormal samples, traditionally requires model retraining for different inlier distribu…
Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video
Xiaohao Xu, Tianyi Zhang, Shibo Zhao +8
We aim to redefine robust ego-motion estimation and photorealistic 3D reconstruction by addressing a critical limitation: the reliance on noise-free data in existing models. While…
SplaTraj: Camera Trajectory Generation with Semantic Gaussian Splatting
Xinyi Liu, Tianyi Zhang, Matthew Johnson-Roberson +1
Many recent developments for robots to represent environments have focused on photorealistic reconstructions. This paper particularly focuses on generating sequences of images from…
PhotoReg: Photometrically Registering 3D Gaussian Splatting Models
Ziwen Yuan, Tianyi Zhang, Matthew Johnson-Roberson +1
Building accurate representations of the environment is critical for intelligent robots to make decisions during deployment. Advances in photorealistic environment models have enab…
Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation
Quanting Xie, So Yeon Min, Pengliang Ji +7
There is no limit to how much a robot might explore and learn, but all of that knowledge needs to be searchable and actionable. Within language research, retrieval augmented genera…
RecGS: Removing Water Caustic with Recurrent Gaussian Splatting
Tianyi Zhang, Weiming Zhi, Kaining Huang +3
Water caustics are commonly observed in seafloor imaging data from shallow-water areas. Traditional methods that remove caustic patterns from images often rely on 2D filtering or p…
3D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objects
Weiming Zhi, Haozhan Tang, Tianyi Zhang +1
Humans have the remarkable ability to use held objects as tools to interact with their environment. For this to occur, humans internally estimate how hand movements affect the obje…
From Perfect to Noisy World Simulation: Customizable Embodied Multi-modal Perturbations for SLAM Robustness Benchmarking
Xiaohao Xu, Tianyi Zhang, Sibo Wang +6
Embodied agents require robust navigation systems to operate in unstructured environments, making the robustness of Simultaneous Localization and Mapping (SLAM) models critical to…
Learning Shared RGB-D Fields: Unified Self-supervised Pre-training for Label-efficient LiDAR-Camera 3D Perception
Xiaohao Xu, Ye Li, Tianyi Zhang +3
Constructing large-scale labeled datasets for multi-modal perception model training in autonomous driving presents significant challenges. This has motivated the development of sel…
Unifying Scene Representation and Hand-Eye Calibration with 3D Foundation Models
Weiming Zhi, Haozhan Tang, Tianyi Zhang +1
Representing the environment is a central challenge in robotics, and is essential for effective decision-making. Traditionally, before capturing images with a manipulator-mounted c…
Teaching Robots Where To Go And How To Act With Human Sketches via Spatial Diagrammatic Instructions
Qilin Sun, Weiming Zhi, Tianyi Zhang +1
This paper introduces Spatial Diagrammatic Instructions (SDIs), an approach for human operators to specify objectives and constraints that are related to spatial regions in the wor…
DarkGS: Learning Neural Illumination and 3D Gaussians Relighting for Robotic Exploration in the Dark
Tianyi Zhang, Kaining Huang, Weiming Zhi +1
Humans have the remarkable ability to construct consistent mental models of an environment, even under limited or varying levels of illumination. We wish to endow robots with this…
Customizable Perturbation Synthesis for Robust SLAM Benchmarking
Xiaohao Xu, Tianyi Zhang, Sibo Wang +6
Robustness is a crucial factor for the successful deployment of robots in unstructured environments, particularly in the domain of Simultaneous Localization and Mapping (SLAM). Sim…
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
Yafei Hu, Quanting Xie, Vidhi Jain +20
Building general-purpose robots that operate seamlessly in any environment, with any object, and utilizing various skills to complete diverse tasks has been a long-standing goal in…
Learning Orbitally Stable Systems for Diagrammatically Teaching
Weiming Zhi, Tianyi Zhang, Matthew Johnson-Roberson
Diagrammatic Teaching is a paradigm for robots to acquire novel skills, whereby the user provides 2D sketches over images of the scene to shape the robot's motion. In this work, we…
Reasoning about the Unseen for Efficient Outdoor Object Navigation
Quanting Xie, Tianyi Zhang, Kedi Xu +2
Robots should exist anywhere humans do: indoors, outdoors, and even unmapped environments. In contrast, the focus of recent advancements in Object Goal Navigation(OGN) has targeted…
Instructing Robots by Sketching: Learning from Demonstration via Probabilistic Diagrammatic Teaching
Weiming Zhi, Tianyi Zhang, Matthew Johnson-Roberson
Learning for Demonstration (LfD) enables robots to acquire new skills by imitating expert demonstrations, allowing users to communicate their instructions in an intuitive manner. R…