Deep Shading: Convolutional Neural Networks for Screen-Space Shading

Abstract

In computer vision, convolutional neural networks (CNNs) have recently achieved new levels of performance for several inverse problems where RGB pixel appearance is mapped to attributes such as positions, normals or reflectance. In computer graphics, screen-space shading has recently increased the visual quality in interactive image synthesis, where per-pixel attributes such as positions, normals or reflectance of a virtual 3D scene are converted into RGB pixel appearance, enabling effects like ambient occlusion, indirect light, scattering, depth-of-field, motion blur, or anti-aliasing. In this paper we consider the diagonal problem: synthesizing appearance from given per-pixel attributes using a CNN. The resulting Deep Shading simulates various screen-space effects at competitive quality and speed while not being programmed by human experts but learned from example images.

References in corpus (7)

• ADADELTA: An Adaptive Learning Rate Method
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Fully Convolutional Networks for Semantic Segmentation
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Deep Convolutional Inverse Graphics Network
• Loss Functions for Neural Networks for Image Processing
• Deep Shading: Convolutional Neural Networks for Screen-Space Shading

Cited by in corpus (28)

• Artificial Intelligence in the Creative Industries: A Review
• Deep Shading: Convolutional Neural Networks for Screen-Space Shading
• Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning
• Self-Supervised Intrinsic Image Decomposition
• Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks
• Neural Light Transport for Relighting and View Synthesis
• Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction
• Net2Vis -- A Visual Grammar for Automatically Generating Publication-Tailored CNN Architecture Visualizations
• Volumetric Isosurface Rendering with Deep Learning-Based Super-Resolution
• RenderNet: A deep convolutional network for differentiable rendering from 3D shapes
• RGB$\leftrightarrow$X: Image decomposition and synthesis using material- and lighting-aware diffusion models
• Relighting Neural Radiance Fields with Shadow and Highlight Hints
• Real-Time Neural Appearance Models
• Gaussian Material Synthesis
• scenery: Flexible Virtual Reality Visualization on the Java VM
• DeProCams: Simultaneous Relighting, Compensation and Shape Reconstruction for Projector-Camera Systems
• Deep Volumetric Ambient Occlusion
• 3D Shape Induction from 2D Views of Multiple Objects
• Deep Illumination: Approximating Dynamic Global Illumination with Generative Adversarial Network
• NeuMIP: Multi-Resolution Neural Materials
• VConv-DAE: Deep Volumetric Shape Learning Without Object Labels
• Learning Neural Light Transport
• Deep and Fast Approximate Order Independent Transparency
• Light Transport Simulation via Generalized Multiple Importance Sampling
• Inferring 3D Shapes from Image Collections using Adversarial Networks
• Neural BRDFs: Representation and Operations
• DeRenderNet: Intrinsic Image Decomposition of Urban Scenes with Shape-(In)dependent Shading Rendering
• Deep Radiance Caching: Convolutional Autoencoders Deeper in Ray Tracing