DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics
arXiv:2005.06223
Abstract
Robots are still limited to controlled conditions, that the robot designer knows with enough details to endow the robot with the appropriate models or behaviors. Learning algorithms add some flexibility with the ability to discover the appropriate behavior given either some demonstrations or a reward to guide its exploration with a reinforcement learning algorithm. Reinforcement learning algorithms rely on the definition of state and action spaces that define reachable behaviors. Their adaptation capability critically depends on the representations of these spaces: small and discrete spaces result in fast learning while large and continuous spaces are challenging and either require a long training period or prevent the robot from converging to an appropriate behavior. Beside the operational cycle of policy execution and the learning cycle, which works at a slower time scale to acquire new policies, we introduce the redescription cycle, a third cycle working at an even slower time scale to generate or adapt the required representations to the robot, its environment and the task. We introduce the challenges raised by this cycle and we present DREAM (Deferred Restructuring of Experience in Autonomous Machines), a developmental cognitive architecture to bootstrap this redescription process stage by stage, build new state representations with appropriate motivations, and transfer the acquired knowledge across domains or tasks or even across robots. We describe results obtained so far with this approach and end up with a discussion of the questions it raises in Neuroscience.
References in corpus (21)
- End to End Learning for Self-Driving Cars
- Robots that can adapt like animals
- Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots
- Data-Efficient Hierarchical Reinforcement Learning
- Stochastic Neural Networks for Hierarchical Reinforcement Learning
- Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning
- Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
- Deep Multi-task Representation Learning: A Tensor Factorisation Approach
- Embodied Evolution in Collective Robotics: A Review
- Multi-Level Discovery of Deep Options
- Curiosity Driven Exploration of Learned Disentangled Goal Spaces
- Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration
- Hierarchical Reinforcement Learning with Hindsight
- DisCoRL: Continual Reinforcement Learning via Policy Distillation
- From exploration to control: learning object manipulation skills through novelty search and local adaptation
- Computational Theories of Curiosity-Driven Learning
- Integration of navigation and action selection functionalities in a computational model of cortico-basal ganglia-thalamo-cortical loops
- S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning
- Learning Robust Options
- Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays
- Bootstrapping Robotic Ecological Perception from a Limited Set of Hypotheses Through Interactive Perception