Improved Part Segmentation Performance by Optimising Realism of Synthetic Images using Cycle Generative Adversarial Networks

Abstract

In this paper we report on improved part segmentation performance using convolutional neural networks to reduce the dependency on the large amount of manually annotated empirical images. This was achieved by optimising the visual realism of synthetic agricultural images.In Part I, a cycle consistent generative adversarial network was applied to synthetic and empirical images with the objective to generate more realistic synthetic images by translating them to the empirical domain. We first hypothesise and confirm that plant part image features such as color and texture become more similar to the empirical domain after translation of the synthetic images.Results confirm this with an improved mean color distribution correlation with the empirical data prior of 0.62 and post translation of 0.90. Furthermore, the mean image features of contrast, homogeneity, energy and entropy moved closer to the empirical mean, post translation. In Part II, 7 experiments were performed using convolutional neural networks with different combinations of synthetic, synthetic translated to empirical and empirical images. We hypothesised that the translated images can be used for (i) improved learning of empirical images, and (ii) that learning without any fine-tuning with empirical images is improved by bootstrapping with translated images over bootstrapping with synthetic images. Results confirm our second and third hypotheses. First a maximum intersection-over-union performance was achieved of 0.52 when bootstrapping with translated images and fine-tuning with empirical images; an 8% increase compared to only using synthetic images. Second, training without any empirical fine-tuning resulted in an average IOU of 0.31; a 55% performance increase over previous methods that only used synthetic images.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Deep Residual Learning for Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Practical recommendations for gradient-based training of deep architectures
• All you need is a good init

Cited by in corpus (6)

• Temporal Prediction and Evaluation of Brassica Growth in the Field using Conditional Generative Adversarial Networks
• Inside Out: Transforming Images of Lab-Grown Plants for Machine Learning Applications in Agriculture
• On the Importance of Visual Context for Data Augmentation in Scene Understanding
• Self-Supervised Data Generation for Precision Agriculture: Blending Simulated Environments with Real Imagery
• DynaDog+T: A Parametric Animal Model for Synthetic Canine Image Generation
• Toward a Procedural Fruit Tree Rendering Framework for Image Analysis