Hypercomplex Image-to-Image Translation

Abstract

Image-to-image translation (I2I) aims at transferring the content representation from an input domain to an output one, bouncing along different target domains. Recent I2I generative models, which gain outstanding results in this task, comprise a set of diverse deep networks each with tens of million parameters. Moreover, images are usually three-dimensional being composed of RGB channels and common neural models do not take dimensions correlation into account, losing beneficial information. In this paper, we propose to leverage hypercomplex algebra properties to define lightweight I2I generative models capable of preserving pre-existing relations among image dimensions, thus exploiting additional input information. On manifold I2I benchmarks, we show how the proposed Quaternion StarGANv2 and parameterized hypercomplex StarGANv2 (PHStarGANv2) reduce parameters and storage memory amount while ensuring high domain translation performance and good image quality as measured by FID and LPIPS scores. Full code is available at: https://github.com/ispamm/HI2I.

References in corpus (11)

• Alias-Free Generative Adversarial Networks
• U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• ReZero is All You Need: Fast Convergence at Large Depth
• Quaternion Convolutional Neural Networks for Heterogeneous Image Processing
• Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events
• PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions
• Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with $1/n$ Parameters
• Quaternion Generative Adversarial Networks
• Hypercomplex-Valued Recurrent Correlation Neural Networks
• Compressing deep quaternion neural networks with targeted regularization

Cited by in corpus (1)

• Quaternion Convolutional Neural Networks: Current Advances and Future Directions