Object Detection Using Sim2Real Domain Randomization for Robotic Applications

Abstract

Robots working in unstructured environments must be capable of sensing and interpreting their surroundings. One of the main obstacles of deep-learning-based models in the field of robotics is the lack of domain-specific labeled data for different industrial applications. In this article, we propose a sim2real transfer learning method based on domain randomization for object detection with which labeled synthetic datasets of arbitrary size and object types can be automatically generated. Subsequently, a state-of-the-art convolutional neural network, YOLOv4, is trained to detect the different types of industrial objects. With the proposed domain randomization method, we could shrink the reality gap to a satisfactory level, achieving 86.32% and 97.38% mAP50 scores, respectively, in the case of zero-shot and one-shot transfers, on our manually annotated dataset containing 190 real images. Our solution fits for industrial use as the data generation process takes less than 0.5 s per image and the training lasts only around 12 h, on a GeForce RTX 2080 Ti GPU. Furthermore, it can reliably differentiate similar classes of objects by having access to only one real image for training. To our best knowledge, this is the only work thus far satisfying these constraints.

arXiv commentPublished in IEEE Transactions on Robotics (T-RO)

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• YOLOv4: Optimal Speed and Accuracy of Object Detection
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Improved Regularization of Convolutional Neural Networks with Cutout
• Semantic Foggy Scene Understanding with Synthetic Data

Cited by in corpus (5)

• Towards Sim-to-Real Industrial Parts Classification with Synthetic Dataset
• A Unification Between Deep-Learning Vision, Compartmental Dynamical Thermodynamics, and Robotic Manipulation for a Circular Economy
• Fully-Synthetic Training for Visual Quality Inspection in Automotive Production
• Sim2Real Grasp Pose Estimation for Adaptive Robotic Applications
• HiER: Highlight Experience Replay for Boosting Off-Policy Reinforcement Learning Agents