Conditional Invertible Neural Networks for Diverse Image-to-Image Translation
arXiv:2105.02104
Abstract
We introduce a new architecture called a conditional invertible neural network (cINN), and use it to address the task of diverse image-to-image translation for natural images. This is not easily possible with existing INN models due to some fundamental limitations. The cINN combines the purely generative INN model with an unconstrained feed-forward network, which efficiently preprocesses the conditioning image into maximally informative features. All parameters of a cINN are jointly optimized with a stable, maximum likelihood-based training procedure. Even though INN-based models have received far less attention in the literature than GANs, they have been shown to have some remarkable properties absent in GANs, e.g. apparent immunity to mode collapse. We find that our cINNs leverage these properties for image-to-image translation, demonstrated on day to night translation and image colorization. Furthermore, we take advantage of our bidirectional cINN architecture to explore and manipulate emergent properties of the latent space, such as changing the image style in an intuitive way.
arXiv commentarXiv admin note: text overlap with arXiv:1907.02392
References in corpus (11)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- NICE: Non-linear Independent Components Estimation
- Toward Multimodal Image-to-Image Translation
- Density estimation using Real NVP
- Semantic Image Synthesis with Spatially-Adaptive Normalization
- Analyzing Inverse Problems with Invertible Neural Networks
- WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
- Excessive Invariance Causes Adversarial Vulnerability
- Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)