Image Synthesis with a Single (Robust) Classifier

Abstract

We show that the basic classification framework alone can be used to tackle some of the most challenging tasks in image synthesis. In contrast to other state-of-the-art approaches, the toolkit we develop is rather minimal: it uses a single, off-the-shelf classifier for all these tasks. The crux of our approach is that we train this classifier to be adversarially robust. It turns out that adversarial robustness is precisely what we need to directly manipulate salient features of the input. Overall, our findings demonstrate the utility of robustness in the broader machine learning context. Code and models for our experiments can be found at https://git.io/robust-apps.

References in corpus (9)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Pixel Recurrent Neural Networks
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Deep Image Prior

Cited by in corpus (25)

• Do Adversarially Robust ImageNet Models Transfer Better?
• Boosting Adversarial Training with Hypersphere Embedding
• Partial success in closing the gap between human and machine vision
• Understanding and Improving Fast Adversarial Training
• Automatic Shortcut Removal for Self-Supervised Representation Learning
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
• The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization
• What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction
• InfoAT: Improving Adversarial Training Using the Information Bottleneck Principle
• Training Robust Deep Neural Networks via Adversarial Noise Propagation
• Poisoned classifiers are not only backdoored, they are fundamentally broken
• Hold me tight! Influence of discriminative features on deep network boundaries
• Overinterpretation reveals image classification model pathologies
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training
• Rethinking the Role of Gradient-Based Attribution Methods for Model Interpretability
• Bowtie Networks: Generative Modeling for Joint Few-Shot Recognition and Novel-View Synthesis
• On the Benefits of Models with Perceptually-Aligned Gradients
• Improving Model Robustness with Latent Distribution Locally and Globally
• Do Input Gradients Highlight Discriminative Features?
• A Variational Inequality Approach to Bayesian Regression Games
• Orthogonal Deep Models As Defense Against Black-Box Attacks
• Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers
• Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More
• Attack to Fool and Explain Deep Networks