TACO: Learning Task Decomposition via Temporal Alignment for Control

Abstract

Many advanced Learning from Demonstration (LfD) methods consider the decomposition of complex, real-world tasks into simpler sub-tasks. By reusing the corresponding sub-policies within and between tasks, they provide training data for each policy from different high-level tasks and compose them to perform novel ones. Existing approaches to modular LfD focus either on learning a single high-level task or depend on domain knowledge and temporal segmentation. In contrast, we propose a weakly supervised, domain-agnostic approach based on task sketches, which include only the sequence of sub-tasks performed in each demonstration. Our approach simultaneously aligns the sketches with the observed demonstrations and learns the required sub-policies. This improves generalisation in comparison to separate optimisation procedures. We evaluate the approach on multiple domains, including a simulated 3D robot arm control task using purely image-based observations. The results show that our approach performs commensurately with fully supervised approaches, while requiring significantly less annotation effort.

arXiv comment12 Pages. Published at ICML 2018

References in corpus (3)

• Sequence Transduction with Recurrent Neural Networks
• Modular Multitask Reinforcement Learning with Policy Sketches
• Multi-Level Discovery of Deep Options

Cited by in corpus (14)

• One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning
• Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning
• Automata Guided Reinforcement Learning With Demonstrations
• CompILE: Compositional Imitation Learning and Execution
• Conditional Driving from Natural Language Instructions
• Data-efficient Hindsight Off-policy Option Learning
• AutoPreview: A Framework for Autopilot Behavior Understanding
• Provable Hierarchical Imitation Learning via EM
• Abstract Demonstrations and Adaptive Exploration for Efficient and Stable Multi-step Sparse Reward Reinforcement Learning
• Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation
• Inverse Contextual Bandits: Learning How Behavior Evolves over Time
• Visual Goal-Directed Meta-Learning with Contextual Planning Networks
• Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation