12 papers
SPEAR: A Simulator for Photorealistic Embodied AI Research
Mike Roberts, Renhan Wang, Rushikesh Zawar +10
Interactive simulators have become powerful tools for training embodied agents and generating synthetic visual data, but existing photorealistic simulators suffer from limited gene…
Robot Self-Improvement via Human-Video Dynamics Models
Hanzhi Chen, Anran Zhang, Simon Schaefer +5
A central question in robot learning is how to acquire skills from the kinds of data that humans learn from: passive observation, embodied practice, and the experience of failure.…
HumanFlow -- Diffusion-Driven MAV Navigation Among Humans via Tightly-Coupled Motion Tracking, Forecasting, and Control
Simon Schaefer, Joshua Näf, Stefan Leutenegger
Robust and accurate perception of humans in their 3D scene context is essential for integrating robots into everyday environments. Existing approaches, however, often fail to predi…
OpenSGA: Efficient 3D Scene Graph Alignment in the Open World
Gang Chen, Sebastián Barbas Laina, Stefan Leutenegger +1
Scene graph alignment establishes object correspondences between two 3D scene graphs constructed from partially overlapping observations. This enables efficient scene understanding…
GOPLA: Generalizable Object Placement Learning via Synthetic Augmentation of Human Arrangement
Yao Zhong, Hanzhi Chen, Simon Schaefer +2
Robots are expected to serve as intelligent assistants, helping humans with everyday household organization. A central challenge in this setting is the task of object placement, wh…
Actron3D: Learning Actionable Neural Functions from Videos for Transferable Robotic Manipulation
Anran Zhang, Hanzhi Chen, Yannick Burkhardt +4
We present Actron3D, a framework that enables robots to acquire transferable 6-DoF manipulation skills from monocular, uncalibrated, RGB-only human demonstration videos. Our key id…
VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation
Hanzhi Chen, Boyang Sun, Anran Zhang +2
Future robots are envisioned as versatile systems capable of performing a variety of household tasks. The big question remains, how can we bridge the embodiment gap while minimizin…
FrontierNet: Learning Visual Cues to Explore
Boyang Sun, Hanzhi Chen, Stefan Leutenegger +3
Exploration of unknown environments is crucial for autonomous robots; it allows them to actively reason and decide on what new data to acquire for different tasks, such as mapping,…
FuncGrasp: Learning Object-Centric Neural Grasp Functions from Single Annotated Example Object
Hanzhi Chen, Binbin Xu, Stefan Leutenegger
We present FuncGrasp, a framework that can infer dense yet reliable grasp configurations for unseen objects using one annotated object and single-view RGB-D observation via categor…
NeRF-VO: Real-Time Sparse Visual Odometry with Neural Radiance Fields
Jens Naumann, Binbin Xu, Stefan Leutenegger +1
We introduce a novel monocular visual odometry (VO) system, NeRF-VO, that integrates learning-based sparse visual odometry for low-latency camera tracking and a neural radiance sce…
Dynamic LiDAR Re-simulation using Compositional Neural Fields
Hanfeng Wu, Xingxing Zuo, Stefan Leutenegger +3
We introduce DyNFL, a novel neural field-based approach for high-fidelity re-simulation of LiDAR scans in dynamic driving scenes. DyNFL processes LiDAR measurements from dynamic en…
Anthropomorphic Grasping with Neural Object Shape Completion
Diego Hidalgo-Carvajal, Hanzhi Chen, Gemma C. Bettelani +6
The progressive prevalence of robots in human-suited environments has given rise to a myriad of object manipulation techniques, in which dexterity plays a paramount role. It is wel…