Alias-Free Generative Adversarial Networks

Abstract

We observe that despite their hierarchical convolutional nature, the synthesis process of typical generative adversarial networks depends on absolute pixel coordinates in an unhealthy manner. This manifests itself as, e.g., detail appearing to be glued to image coordinates instead of the surfaces of depicted objects. We trace the root cause to careless signal processing that causes aliasing in the generator network. Interpreting all signals in the network as continuous, we derive generally applicable, small architectural changes that guarantee that unwanted information cannot leak into the hierarchical synthesis process. The resulting networks match the FID of StyleGAN2 but differ dramatically in their internal representations, and they are fully equivariant to translation and rotation even at subpixel scales. Our results pave the way for generative models better suited for video and animation.

References in corpus (11)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Self-Attention Generative Adversarial Networks
• Zero-Shot Text-to-Image Generation
• Group Equivariant Convolutional Networks
• Making Convolutional Networks Shift-Invariant Again
• General $E(2)$-Equivariant Steerable CNNs
• How Much Position Information Do Convolutional Neural Networks Encode?
• Spatially Controllable Image Synthesis with Internal Representation Collaging
• SWAGAN: A Style-based Wavelet-driven Generative Model

Cited by in corpus (62)

• Blended Diffusion for Text-driven Editing of Natural Images
• StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
• A survey on GANs for computer vision: Recent research, analysis and taxonomy
• Face Generation and Editing with StyleGAN: A Survey
• Physical Adversarial Attack meets Computer Vision: A Decade Survey
• Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
• Deep Learning Based Single Sample Per Person Face Recognition: A Survey
• CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis
• Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics
• FFNeRV: Flow-Guided Frame-Wise Neural Representations for Videos
• Copyright Protection and Accountability of Generative AI:Attack, Watermarking and Attribution
• The Free Energy Principle for Perception and Action: A Deep Learning Perspective
• Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Survey
• Artificial intelligence approaches for materials-by-design of energetic materials: state-of-the-art, challenges, and future directions
• GANSlider: How Users Control Generative Models for Images using Multiple Sliders with and without Feedforward Information
• Evaluating the Performance of StyleGAN2-ADA on Medical Images
• Heterogeneous Face Recognition via Face Synthesis with Identity-Attribute Disentanglement
• FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models
• FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes
• Talking Head from Speech Audio using a Pre-trained Image Generator
• Guided Co-Modulated GAN for 360° Field of View Extrapolation
• Human Motion Transfer with 3D Constraints and Detail Enhancement
• Controllable Generation with Text-to-Image Diffusion Models: A Survey
• FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold
• Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data
• Dense Pixel-to-Pixel Harmonization via Continuous Image Representation
• Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks
• CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
• Iterative $\alpha$-(de)Blending: a Minimalist Deterministic Diffusion Model
• Neural Implicit Morphing of Face Images
• ChildGAN: Large Scale Synthetic Child Facial Data Using Domain Adaptation in StyleGAN
• Hypercomplex Image-to-Image Translation
• MotionVideoGAN: A Novel Video Generator Based on the Motion Space Learned from Image Pairs
• Probabilistic Precision and Recall Towards Reliable Evaluation of Generative Models
• PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mapping
• Translation-Enhanced Multilingual Text-to-Image Generation
• Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activations
• FACEMUG: A Multimodal Generative and Fusion Framework for Local Facial Editing
• Domain-Scalable Unpaired Image Translation via Latent Space Anchoring
• Rethinking Implicit Neural Representations for Vision Learners
• on the effectiveness of generative adversarial network on anomaly detection
• Methodology for generating synthetic labeled datasets for visual container inspection
• Neural Fields in Visual Computing and Beyond
• Evaluating the Efficacy of Skincare Product: A Realistic Short-Term Facial Pore Simulation
• KBody: Towards general, robust, and aligned monocular whole-body estimation
• Content-Aware Preserving Image Generation
• Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesis
• dCoNNear: An Artifact-Free Neural Network Architecture for Closed-loop Audio Signal Processing
• DIFAI: Diverse Facial Inpainting using StyleGAN Inversion
• Adaptive Input-image Normalization for Solving the Mode Collapse Problem in GAN-based X-ray Images
• Controlling the Output of a Generative Model by Latent Feature Vector Shifting
• EGAIN: Extended GAn INversion
• BOSC: A Backdoor-based Framework for Open Set Synthetic Image Attribution
• Synthesis of pulses from particle detectors with a Generative Adversarial Network (GAN)
• DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editing
• ComFace: Facial Representation Learning with Synthetic Data for Comparing Faces
• Generating Diverse Indoor Furniture Arrangements
• Learning Gaze-aware Compositional GAN
• 3D-Aware Semantic-Guided Generative Model for Human Synthesis
• CYBORG: Blending Human Saliency Into the Loss Improves Deep Learning
• SAT3D: Image-driven Semantic Attribute Transfer in 3D
• Discriminator Synthesis: On reusing the other half of Generative Adversarial Networks