Learning Robotic Navigation from Experience: Principles, Methods, and Recent Results
arXiv:2212.06759 · doi:10.1098/rstb.2021.0447
Abstract
Navigation is one of the most heavily studied problems in robotics, and is conventionally approached as a geometric mapping and planning problem. However, real-world navigation presents a complex set of physical challenges that defies simple geometric abstractions. Machine learning offers a promising way to go beyond geometry and conventional planning, allowing for navigational systems that make decisions based on actual prior experience. Such systems can reason about traversability in ways that go beyond geometry, accounting for the physical outcomes of their actions and exploiting patterns in real-world environments. They can also improve as more data is collected, potentially providing a powerful network effect. In this article, we present a general toolkit for experiential learning of robotic navigation skills that unifies several recent approaches, describe the underlying design principles, summarize experimental results from several of our recent papers, and discuss open problems and directions for future work.
arXiv commentFinal print version is here: https://royalsocietypublishing.org/doi/10.1098/rstb.2021.0447
References in corpus (34)
- Continuous control with deep reinforcement learning
- Representation Learning with Contrastive Predictive Coding
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
- Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
- On Calibration of Modern Neural Networks
- Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
- End-to-End Training of Deep Visuomotor Policies
- Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
- Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
- Conservative Q-Learning for Offline Reinforcement Learning
- Visual Reinforcement Learning with Imagined Goals
- Learning to Explore using Active Neural SLAM
- Differentiable MPC for End-to-end Planning and Control
- Driving Policy Transfer via Modularity and Abstraction
- Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning
- Neural SLAM: Learning to Explore with External Memory
- Critic Regularized Regression
- Safe Robot Navigation via Multi-Modal Anomaly Detection
- ViNG: Learning Open-World Navigation with Visual Goals
- Gibson Env: Real-World Perception for Embodied Agents
- ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints
- Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
- Many-Goals Reinforcement Learning
- Goal-Conditioned Reinforcement Learning with Imagined Subgoals
- Broadly-Exploring, Local-Policy Trees for Long-Horizon Task Planning
- Rapid Exploration for Open-World Navigation with Latent Goal Models
- Sparse Graphical Memory for Robust Planning
- To Go or Not To Go? A Near Unsupervised Learning Approach For Robot Navigation
- Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL
- C-Learning: Learning to Achieve Goals via Recursive Classification
- Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey
- Hierarchically Integrated Models: Learning to Navigate from Heterogeneous Robots