Optimal Transport driven CycleGAN for Unsupervised Learning in Inverse Problems

Abstract

To improve the performance of classical generative adversarial network (GAN), Wasserstein generative adversarial networks (W-GAN) was developed as a Kantorovich dual formulation of the optimal transport (OT) problem using Wasserstein-1 distance. However, it was not clear how cycleGAN-type generative models can be derived from the optimal transport theory. Here we show that a novel cycleGAN architecture can be derived as a Kantorovich dual OT formulation if a penalized least square (PLS) cost with deep learning-based inverse path penalty is used as a transportation cost. One of the most important advantages of this formulation is that depending on the knowledge of the forward problem, distinct variations of cycleGAN architecture can be derived: for example, one with two pairs of generators and discriminators, and the other with only a single pair of generator and discriminator. Even for the two generator cases, we show that the structural knowledge of the forward operator can lead to a simpler generator architecture which significantly simplifies the neural network training. The new cycleGAN formulation, what we call the OT-cycleGAN, have been applied for various biomedical imaging problems, such as accelerated magnetic resonance imaging (MRI), super-resolution microscopy, and low-dose x-ray computed tomography (CT). Experimental results confirm the efficacy and flexibility of the theory.

accepted for publication in the SIAM Journal on Imaging Sciences

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Spectral Normalization for Generative Adversarial Networks
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)
• Deep-STORM: super-resolution single-molecule microscopy by deep learning
• fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
• Compressed Sensing using Generative Models
• Cycle Consistent Adversarial Denoising Network for Multiphase Coronary CT Angiography
• Compressed Sensing with Deep Image Prior and Learned Regularization
• Deep Compressed Sensing
• Attribute-Guided Face Generation Using Conditional CycleGAN

Cited by in corpus (7)

• Multi-Mask Self-Supervised Learning for Physics-Guided Neural Networks in Highly Accelerated MRI
• Continuous Conversion of CT Kernel using Switchable CycleGAN with AdaIN
• Unsupervised MR Motion Artifact Deep Learning using Outlier-Rejecting Bootstrap Aggregation
• Unsupervised CT Metal Artifact Learning using Attention-guided beta-CycleGAN
• Reliable Multi-modal Medical Image-to-image Translation Independent of Pixel-wise Aligned Data
• Unpaired Deep Learning for Accelerated MRI using Optimal Transport Driven CycleGAN
• Unsupervised Deep Learning for MR Angiography with Flexible Temporal Resolution