N${}^2$M${}^2$: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environments

Abstract

Despite its importance in both industrial and service robotics, mobile manipulation remains a significant challenge as it requires a seamless integration of end-effector trajectory generation with navigation skills as well as reasoning over long-horizons. Existing methods struggle to control the large configuration space, and to navigate dynamic and unknown environments. In previous work, we proposed to decompose mobile manipulation tasks into a simplified motion generator for the end-effector in task space and a trained reinforcement learning agent for the mobile base to account for kinematic feasibility of the motion. In this work, we introduce Neural Navigation for Mobile Manipulation (N${}^2$M${}^2$) which extends this decomposition to complex obstacle environments and enables it to tackle a broad range of tasks in real world settings. The resulting approach can perform unseen, long-horizon tasks in unexplored environments while instantly reacting to dynamic obstacles and environmental changes. At the same time, it provides a simple way to define new mobile manipulation tasks. We demonstrate the capabilities of our proposed approach in extensive simulation and real-world experiments on multiple kinematically diverse mobile manipulators. Code and videos are publicly available at http://mobile-rl.cs.uni-freiburg.de.

Project website: http://mobile-rl.cs.uni-freiburg.de; Accepted at T-RO

References in corpus (7)

• Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study
• Rearrangement: A Challenge for Embodied AI
• Robot Learning of Mobile Manipulation with Reachability Behavior Priors
• BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments
• From Learning to Relearning: A Framework for Diminishing Bias in Social Robot Navigation
• Whole-Body Control of a Mobile Manipulator using End-to-End Reinforcement Learning
• Predicting Dense and Context-aware Cost Maps for Semantic Robot Navigation

Cited by in corpus (6)

• Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulation
• Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation
• The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation
• Whole-Body Teleoperation for Mobile Manipulation at Zero Added Cost
• BaSeNet: A Learning-based Mobile Manipulator Base Pose Sequence Planning for Pickup Tasks
• Task-Driven Co-Design of Mobile Manipulators