Face Reconstruction with Variational Autoencoder and Face Masks
arXiv:2112.02139 · doi:10.5753/eniac.2021.18282
Abstract
Variational AutoEncoders (VAE) employ deep learning models to learn a continuous latent z-space that is subjacent to a high-dimensional observed dataset. With that, many tasks are made possible, including face reconstruction and face synthesis. In this work, we investigated how face masks can help the training of VAEs for face reconstruction, by restricting the learning to the pixels selected by the face mask. An evaluation of the proposal using the celebA dataset shows that the reconstructed images are enhanced with the face masks, especially when SSIM loss is used either with l1 or l2 loss functions. We noticed that the inclusion of a decoder for face mask prediction in the architecture affected the performance for l1 or l2 loss functions, while this was not the case for the SSIM loss. Besides, SSIM perceptual loss yielded the crispest samples between all hypotheses tested, although it shifts the original color of the image, making the usage of the l1 or l2 losses together with SSIM helpful to solve this issue.
arXiv comment12 pages, 7 figures, 18th Encontro Nacional de Inteligência Artificial e Computacional (ENIAC)
References in corpus (11)
- An Introduction to Variational Autoencoders
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- Which Training Methods for GANs do actually Converge?
- Autoencoding beyond pixels using a learned similarity metric
- NVAE: A Deep Hierarchical Variational Autoencoder
- Generating Images with Perceptual Similarity Metrics based on Deep Networks
- Understanding SSIM
- AttGAN: Facial Attribute Editing by Only Changing What You Want
- Make a Face: Towards Arbitrary High Fidelity Face Manipulation
- A Novel Just-Noticeable-Difference-based Saliency-Channel Attention Residual Network for Full-Reference Image Quality Predictions