Decompose to manipulate: Manipulable Object Synthesis in 3D Medical Images with Structured Image Decomposition
arXiv:1812.01737
Abstract
The performance of medical image analysis systems is constrained by the quantity of high-quality image annotations. Such systems require data to be annotated by experts with years of training, especially when diagnostic decisions are involved. Such datasets are thus hard to scale up. In this context, it is hard for supervised learning systems to generalize to the cases that are rare in the training set but would be present in real-world clinical practices. We believe that the synthetic image samples generated by a system trained on the real data can be useful for improving the supervised learning tasks in the medical image analysis applications. Allowing the image synthesis to be manipulable could help synthetic images provide complementary information to the training data rather than simply duplicating the real-data manifold. In this paper, we propose a framework for synthesizing 3D objects, such as pulmonary nodules, in 3D medical images with manipulable properties. The manipulation is enabled by decomposing of the object of interests into its segmentation mask and a 1D vector containing the residual information. The synthetic object is refined and blended into the image context with two adversarial discriminators. We evaluate the proposed framework on lung nodules in 3D chest CT images and show that the proposed framework could generate realistic nodules with manipulable shapes, textures and locations, etc. By sampling from both the synthetic nodules and the real nodules from 2800 3D CT volumes during the classifier training, we show the synthetic patches could improve the overall nodule detection performance by average 8.44% competition performance metric (CPM) score.
References in corpus (17)
- Spectral Normalization for Generative Adversarial Networks
- The Kinetics Human Action Video Dataset
- Point-Set Registration: Coherent Point Drift
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification
- On the Convergence of Adam and Beyond
- Improved Training of Wasserstein GANs
- Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge
- Disentangling by Factorising
- Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?
- Learning Hierarchical Semantic Image Manipulation through Structured Representations
- High-Resolution Mammogram Synthesis using Progressive Generative Adversarial Networks
- Disentangled Person Image Generation
- Conditional Infilling GANs for Data Augmentation in Mammogram Classification
- Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks
- ScarGAN: Chained Generative Adversarial Networks to Simulate Pathological Tissue on Cardiovascular MR Scans
- Natural and Effective Obfuscation by Head Inpainting
Cited by in corpus (7)
- A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
- Learning Disentangled Representations in the Imaging Domain
- 3D Tomographic Pattern Synthesis for Enhancing the Quantification of COVID-19
- Class-Aware Adversarial Lung Nodule Synthesis in CT Images
- No Surprises: Training Robust Lung Nodule Detection for Low-Dose CT Scans by Augmenting with Adversarial Attacks
- Correlation via synthesis: end-to-end nodule image generation and radiogenomic map learning based on generative adversarial network
- ASIST: Annotation-free Synthetic Instance Segmentation and Tracking by Adversarial Simulations