FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy

Abstract

With the massive proliferation of data-driven algorithms, such as deep learning-based approaches, the availability of high-quality data is of great interest. Volumetric data is very important in medicine, as it ranges from disease diagnoses to therapy monitoring. When the dataset is sufficient, models can be trained to help doctors with these tasks. Unfortunately, there are scenarios where large amounts of data is unavailable. For example, rare diseases and privacy issues can lead to restricted data availability. In non-medical fields, the high cost of obtaining enough high-quality data can also be a concern. A solution to these problems can be the generation of realistic synthetic data using Generative Adversarial Networks (GANs). The existence of these mechanisms is a good asset, especially in healthcare, as the data must be of good quality, realistic, and without privacy issues. Therefore, most of the publications on volumetric GANs are within the medical domain. In this review, we provide a summary of works that generate realistic volumetric synthetic data using GANs. We therefore outline GAN-based methods in these areas with common architectures, loss functions and evaluation metrics, including their advantages and disadvantages. We present a novel taxonomy, evaluations, challenges, and research opportunities to provide a holistic overview of the current state of volumetric GANs.

arXiv comment88 pages

References in corpus (26)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Automated Design of Deep Learning Methods for Biomedical Image Segmentation
• Conditional Generative Adversarial Nets
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Diffusion Models Beat GANs on Image Synthesis
• Generative Adversarial Network in Medical Imaging: A Review
• Diffusion Models in Vision: A Survey
• Mean Absolute Percentage Error for regression models
• Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge
• Training Generative Adversarial Networks with Limited Data
• Reconstruction of three-dimensional porous media using generative adversarial neural networks
• The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
• Med3D: Transfer Learning for 3D Medical Image Analysis
• RGB-D Salient Object Detection: A Survey
• Generating 3D structures from a 2D slice with GAN-based dimensionality expansion
• Medical Deep Learning -- A systematic Meta-Review
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Generative Adversarial Networks (GANs) in Networking: A Comprehensive Survey & Evaluation
• TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network
• Three Dimensional Fluorescence Microscopy Image Synthesis and Segmentation
• Multi-chart Generative Surface Modeling
• DeepSketchHair: Deep Sketch-based 3D Hair Modeling
• Inflating 2D Convolution Weights for Efficient Generation of 3D Medical Images
• Neuromorphologicaly-preserving Volumetric data encoding using VQ-VAE

Cited by in corpus (1)

• WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesis