Transfer Learning From Synthetic To Real Images Using Variational Autoencoders For Precise Position Detection

Abstract

Capturing and labeling camera images in the real world is an expensive task, whereas synthesizing labeled images in a simulation environment is easy for collecting large-scale image data. However, learning from only synthetic images may not achieve the desired performance in the real world due to a gap between synthetic and real images. We propose a method that transfers learned detection of an object position from a simulation environment to the real world. This method uses only a significantly limited dataset of real images while leveraging a large dataset of synthetic images using variational autoencoders. Additionally, the proposed method consistently performed well in different lighting conditions, in the presence of other distractor objects, and on different backgrounds. Experimental results showed that it achieved accuracy of 1.5mm to 3.5mm on average. Furthermore, we showed how the method can be used in a real-world scenario like a "pick-and-place" robotic task.

arXiv commentCopyright 2018 IEEE - Accepted at ICIP 2018, Athens, Greece, October 7-10, 2018. Video: https://youtu.be/30vji7nJibA. arXiv admin note: text overlap with arXiv:1709.06762

Cited by in corpus (6)

• An Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance Detection
• VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control
• Skill Transfer in Deep Reinforcement Learning under Morphological Heterogeneity
• 3D Convolution on RGB-D Point Clouds for Accurate Model-free Object Pose Estimation
• Deep Reinforcement Learning-Aided Random Access
• SyDog: A Synthetic Dog Dataset for Improved 2D Pose Estimation