Active Learning in Robotics: A Review of Control Principles
arXiv:2106.13697 · doi:10.1016/j.mechatronics.2021.102576
Abstract
Active learning is a decision-making process. In both abstract and physical settings, active learning demands both analysis and action. This is a review of active learning in robotics, focusing on methods amenable to the demands of embodied learning systems. Robots must be able to learn efficiently and flexibly through continuous online deployment. This poses a distinct set of control-oriented challenges -- one must choose suitable measures as objectives, synthesize real-time control, and produce analyses that guarantee performance and safety with limited knowledge of the environment or robot itself. In this work, we survey the fundamental components of robotic active learning systems. We discuss classes of learning tasks that robots typically encounter, measures with which they gauge the information content of observations, and algorithms for generating action plans. Moreover, we provide a variety of examples -- from environmental mapping to nonparametric shape estimation -- that highlight the qualitative differences between learning tasks, information measures, and control techniques. We conclude with a discussion of control-oriented open challenges, including safety-constrained learning and distributed learning.
arXiv comment25 pages
References in corpus (43)
- Continuous control with deep reinforcement learning
- Federated Learning: Challenges, Methods, and Future Directions
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
- Control Barrier Function Based Quadratic Programs for Safety Critical Systems
- A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
- On Dynamic Mode Decomposition: Theory and Applications
- Deep learning for universal linear embeddings of nonlinear dynamics
- Experimental Comparison of Open Source Visual-Inertial-Based State Estimation Algorithms in the Underwater Domain
- Recurrent Models of Visual Attention
- Benchmarking Deep Reinforcement Learning for Continuous Control
- Gaussian Processes for Data-Efficient Learning in Robotics and Control
- Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control
- How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
- A Terradynamics of Legged Locomotion on Granular Media
- Reinforcement Learning with Deep Energy-Based Policies
- An Algorithmic Perspective on Imitation Learning
- Generative Adversarial Imitation Learning
- Safe Model-based Reinforcement Learning with Stability Guarantees
- Path integrals and symmetry breaking for optimal control theory
- Efficient Informative Sensing using Multiple Robots
- Active Learning with Statistical Models
- InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
- Ergodic Exploration of Distributed Information
- Sim-to-Real Robot Learning from Pixels with Progressive Nets
- Efficient Multi-Robot Coverage of a Known Environment
- Multi-Robot Informative Path Planning for Active Sensing of Environmental Phenomena: A Tale of Two Algorithms
- Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction
- The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems
- Learning Stable Deep Dynamics Models
- Decentralized Ergodic Control: Distribution-Driven Sensing and Exploration for Multi-Agent Systems
- Learning Dexterous Manipulation Policies from Experience and Imitation
- Trajectory Synthesis for Fisher Information Maximization
- Model-Based Active Source Identification in Complex Environments
- Ergodic Exploration using Binary Sensing for Non-Parametric Shape Estimation
- Maximum Entropy RL (Provably) Solves Some Robust RL Problems
- Learning Stability Certificates from Data
- A data-efficient geometrically inspired polynomial kernel for robot inverse dynamics
- Active Imitation Learning via Reduction to I.I.D. Active Learning
- Data-Driven Measurement Models for Active Localization in Sparse Environments
- Dynamical System Segmentation for Information Measures in Motion
- Learning Stable Models for Prediction and Control
- Self-organized novelty detection in driven spin glasses