HyperInverter: Improving StyleGAN Inversion via Hypernetwork

Abstract

Real-world image manipulation has achieved fantastic progress in recent years as a result of the exploration and utilization of GAN latent spaces. GAN inversion is the first step in this pipeline, which aims to map the real image to the latent code faithfully. Unfortunately, the majority of existing GAN inversion methods fail to meet at least one of the three requirements listed below: high reconstruction quality, editability, and fast inference. We present a novel two-phase strategy in this research that fits all requirements at the same time. In the first phase, we train an encoder to map the input image to StyleGAN2 $\mathcal{W}$-space, which was proven to have excellent editability but lower reconstruction quality. In the second phase, we supplement the reconstruction ability in the initial phase by leveraging a series of hypernetworks to recover the missing information during inversion. These two steps complement each other to yield high reconstruction quality thanks to the hypernetwork branch and excellent editability due to the inversion done in the $\mathcal{W}$-space. Our method is entirely encoder-based, resulting in extremely fast inference. Extensive experiments on two challenging datasets demonstrate the superiority of our method.

arXiv commentAccepted to CVPR 2022; Project page is located at https://di-mi-ta.github.io/HyperInverter/

References in corpus (9)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Deep Learning using Rectified Linear Units (ReLU)
• Improved Baselines with Momentum Contrastive Learning
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Which Training Methods for GANs do actually Converge?
• Understanding the Role of Individual Units in a Deep Neural Network
• Graph HyperNetworks for Neural Architecture Search
• Collaborative Learning for Faster StyleGAN Embedding
• LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions