Semantically Robust Unpaired Image Translation for Data with Unmatched Semantics Statistics
arXiv:2012.04932
Abstract
Many applications of unpaired image-to-image translation require the input contents to be preserved semantically during translations. Unaware of the inherently unmatched semantics distributions between source and target domains, existing distribution matching methods (i.e., GAN-based) can give undesired solutions. In particular, although producing visually reasonable outputs, the learned models usually flip the semantics of the inputs. To tackle this without using extra supervision, we propose to enforce the translated outputs to be semantically invariant w.r.t. small perceptual variations of the inputs, a property we call "semantic robustness". By optimizing a robustness loss w.r.t. multi-scale feature space perturbations of the inputs, our method effectively reduces semantics flipping and produces translations that outperform existing methods both quantitatively and qualitatively.
arXiv commentAccepted to ICCV 2021
References in corpus (22)
- Rethinking Atrous Convolution for Semantic Image Segmentation
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Generative Adversarial Text to Image Synthesis
- BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
- Data Augmentation Generative Adversarial Networks
- Domain Separation Networks
- Unsupervised Cross-Domain Image Generation
- Unsupervised Attention-guided Image to Image Translation
- Robust Large Margin Deep Neural Networks
- Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data
- CycleGAN, a Master of Steganography
- Predicting the Generalization Gap in Deep Networks with Margin Distributions
- Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach
- Improving Generalization and Stability of Generative Adversarial Networks
- Norm-Based Capacity Control in Neural Networks
- Robustness of Conditional GANs to Noisy Labels
- On the Discrimination-Generalization Tradeoff in GANs
- AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks
- Adversarial Self-Defense for Cycle-Consistent GANs
- Harmonic Unpaired Image-to-image Translation
- One-pixel Signature: Characterizing CNN Models for Backdoor Detection
- Information-Theoretic Local Minima Characterization and Regularization