Active choice of teachers, learning strategies and goals for a socially guided intrinsic motivation learner

Abstract

We present an active learning architecture that allows a robot to actively learn which data collection strategy is most efficient for acquiring motor skills to achieve multiple outcomes, and generalise over its experience to achieve new outcomes. The robot explores its environment both via interactive learning and goal-babbling. It learns at the same time when, who and what to actively imitate from several available teachers, and learns when not to use social guidance but use active goal-oriented self-exploration. This is formalised in the framework of life-long strategic learning. The proposed architecture, called Socially Guided Intrinsic Motivation with Active Choice of Teacher and Strategy (SGIM-ACTS), relies on hierarchical active decisions of what and how to learn driven by empirical evaluation of learning progress for each learning strategy. We illustrate with an experiment where a simulated robot learns to control its arm for realising two kinds of different outcomes. It has to choose actively and hierarchically at each learning episode: 1) what to learn: which outcome is most interesting to select as a goal to focus on for goal-directed exploration; 2) how to learn: which data collection strategy to use among self-exploration, mimicry and emulation; 3) once he has decided when and what to imitate by choosing mimicry or emulation, then he has to choose who to imitate, from a set of different teachers. We show that SGIM-ACTS learns significantly more efficiently than using single learning strategies, and coherently selects the best strategy with respect to the chosen outcome, taking advantage of the available teachers (with different levels of skills).

References in corpus (1)

• Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots

Cited by in corpus (9)

• Socially Guided Intrinsic Motivation for Robot Learning of Motor Skills
• CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning
• Accuracy-based Curriculum Learning in Deep Reinforcement Learning
• Autonomous development and learning in artificial intelligence and robotics: Scaling up deep learning to human--like learning
• Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchy
• Learning a Set of Interrelated Tasks by Using Sequences of Motor Policies for a Strategic Intrinsically Motivated Learner
• Robots Learn Increasingly Complex Tasks with Intrinsic Motivation and Automatic Curriculum Learning
• Diversity-Driven Selection of Exploration Strategies in Multi-Armed Bandits
• Open challenges in understanding development and evolution of speech forms: The roles of embodied self-organization, motivation and active exploration