Few-Shot Image-to-Semantics Translation for Policy Transfer in Reinforcement Learning
arXiv:2301.13343 · doi:10.1109/IJCNN55064.2022.9892464
Abstract
We investigate policy transfer using image-to-semantics translation to mitigate learning difficulties in vision-based robotics control agents. This problem assumes two environments: a simulator environment with semantics, that is, low-dimensional and essential information, as the state space, and a real-world environment with images as the state space. By learning mapping from images to semantics, we can transfer a policy, pre-trained in the simulator, to the real world, thereby eliminating real-world on-policy agent interactions to learn, which are costly and risky. In addition, using image-to-semantics mapping is advantageous in terms of the computational efficiency to train the policy and the interpretability of the obtained policy over other types of sim-to-real transfer strategies. To tackle the main difficulty in learning image-to-semantics mapping, namely the human annotation cost for producing a training dataset, we propose two techniques: pair augmentation with the transition function in the simulator environment and active learning. We observed a reduction in the annotation cost without a decline in the performance of the transfer, and the proposed approach outperformed the existing approach without annotation.
arXiv commentThe 2022 International Joint Conference on Neural Networks (IJCNN2022)
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Auto-Encoding Variational Bayes
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
- DeepMind Control Suite
- Recurrent World Models Facilitate Policy Evolution
- Learning by Playing - Solving Sparse Reward Tasks from Scratch
- Driving Policy Transfer via Modularity and Abstraction
- An Optimistic Perspective on Offline Reinforcement Learning
- Diverse mini-batch Active Learning
- Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation
- Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation