Interactive Task Encoding System for Learning-from-Observation
arXiv:2212.10787 · doi:10.1109/AIM46323.2023.10196126
Abstract
We present the Interactive Task Encoding System (ITES) for teaching robots to perform manipulative tasks. ITES is designed as an input system for the Learning-from-Observation (LfO) framework, which enables household robots to be programmed using few-shot human demonstrations without the need for coding. In contrast to previous LfO systems that rely solely on visual demonstrations, ITES leverages both verbal instructions and interaction to enhance recognition robustness, thus enabling multimodal LfO. ITES identifies tasks from verbal instructions and extracts parameters from visual demonstrations. Meanwhile, the recognition result was reviewed by the user for interactive correction. Evaluations conducted on a real robot demonstrate the successful teaching of multiple operations for several scenarios, suggesting the usefulness of ITES for multimodal LfO. The source code is available at https://github.com/microsoft/symbolic-robot-teaching-interface.
6 pages, 9 figures. Submitted to and accepted by 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM). Last updated April 28st, 2023
References in corpus (8)
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application
- BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
- Grasp-type Recognition Leveraging Object Affordance
- Task-sequencing Simulator: Integrated Machine Learning to Execution Simulation for Robot Manipulation
- Learning-from-Observation System Considering Hardware-Level Reusability
- Task-grasping from human demonstration
- Applying Learning-from-observation to household service robots: three common-sense formulation