Generating Images with Perceptual Similarity Metrics based on Deep Networks

Abstract

Image-generating machine learning models are typically trained with loss functions based on distance in the image space. This often leads to over-smoothed results. We propose a class of loss functions, which we call deep perceptual similarity metrics (DeePSiM), that mitigate this problem. Instead of computing distances in the image space, we compute distances between image features extracted by deep neural networks. This metric better reflects perceptually similarity of images and thus leads to better results. We show three applications: autoencoder training, a modification of a variational autoencoder, and inversion of deep convolutional networks. In all cases, the generated images look sharp and resemble natural images.

arXiv commentminor corrections

Cited by in corpus (28)

• Deep Image Prior
• Adversarially Learned Inference
• InfoVAE: Information Maximizing Variational Autoencoders
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Image De-raining Using a Conditional Generative Adversarial Network
• Neural Photo Editing with Introspective Adversarial Networks
• Mode Regularized Generative Adversarial Networks
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Toward Controlled Generation of Text
• Boosting Resolution and Recovering Texture of micro-CT Images with Deep Learning
• ControlMat: A Controlled Generative Approach to Material Capture
• The PAU Survey: Photometric redshifts using transfer learning from simulations
• Encoding large scale cosmological structure with Generative Adversarial Networks
• Adversarial Message Passing For Graphical Models
• Improving Sampling from Generative Autoencoders with Markov Chains
• Improving Neural Silent Speech Interface Models by Adversarial Training
• Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
• TM-NET: Deep Generative Networks for Textured Meshes
• LaMD: Latent Motion Diffusion for Image-Conditional Video Generation
• Human perception in computer vision
• Darker than Black-Box: Face Reconstruction from Similarity Queries
• Enforcing Perceptual Consistency on Generative Adversarial Networks by Using the Normalised Laplacian Pyramid Distance
• Face Reconstruction with Variational Autoencoder and Face Masks
• Multi-modal data generation with a deep metric variational autoencoder
• MONCAE: Multi-Objective Neuroevolution of Convolutional Autoencoders
• Updating the generator in PPGN-h with gradients flowing through the encoder
• Synthesizing Photorealistic Images with Deep Generative Learning
• Learning Similarity Metrics for Volumetric Simulations with Multiscale CNNs