Stay Positive: Non-Negative Image Synthesis for Augmented Reality

Abstract

In applications such as optical see-through and projector augmented reality, producing images amounts to solving non-negative image generation, where one can only add light to an existing image. Most image generation methods, however, are ill-suited to this problem setting, as they make the assumption that one can assign arbitrary color to each pixel. In fact, naive application of existing methods fails even in simple domains such as MNIST digits, since one cannot create darker pixels by adding light. We know, however, that the human visual system can be fooled by optical illusions involving certain spatial configurations of brightness and contrast. Our key insight is that one can leverage this behavior to produce high quality images with negligible artifacts. For example, we can create the illusion of darker patches by brightening surrounding pixels. We propose a novel optimization procedure to produce images that satisfy both semantic and non-negativity constraints. Our approach can incorporate existing state-of-the-art methods, and exhibits strong performance in a variety of tasks including image-to-image translation and style transfer.

References in corpus (16)

• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Variational Inference with Normalizing Flows
• A Learned Representation For Artistic Style
• Density estimation using Real NVP
• HDR image reconstruction from a single exposure using deep CNNs
• NVAE: A Deep Hierarchical Variational Autoencoder
• A Haar Wavelet-Based Perceptual Similarity Index for Image Quality Assessment
• Unsupervised Attention-guided Image to Image Translation
• Fast Patch-based Style Transfer of Arbitrary Style
• GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
• Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis
• Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
• Example-Guided Image Synthesis across Arbitrary Scenes using Masked Spatial-Channel Attention and Self-Supervision