A Generative Model for Volume Rendering

Abstract

We present a technique to synthesize and analyze volume-rendered images using generative models. We use the Generative Adversarial Network (GAN) framework to compute a model from a large collection of volume renderings, conditioned on (1) viewpoint and (2) transfer functions for opacity and color. Our approach facilitates tasks for volume analysis that are challenging to achieve using existing rendering techniques such as ray casting or texture-based methods. We show how to guide the user in transfer function editing by quantifying expected change in the output image. Additionally, the generative model transforms transfer functions into a view-invariant latent space specifically designed to synthesize volume-rendered images. We use this space directly for rendering, enabling the user to explore the space of volume-rendered images. As our model is independent of the choice of volume rendering process, we show how to analyze volume-rendered images produced by direct and global illumination lighting, for a variety of volume datasets.

References in corpus (1)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Cited by in corpus (14)

• A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization
• InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations
• Volumetric Isosurface Rendering with Deep Learning-Based Super-Resolution
• FrankenGAN: Guided Detail Synthesis for Building Mass-Models Using Style-Synchonized GANs
• Fast Neural Representations for Direct Volume Rendering
• Robust Reference Frame Extraction from Unsteady 2D Vector Fields with Convolutional Neural Networks
• Differentiable Direct Volume Rendering
• Learning Adaptive Sampling and Reconstruction for Volume Visualization
• Deep Volumetric Ambient Occlusion
• Exploratory Lagrangian-Based Particle Tracing Using Deep Learning
• IDLat: An Importance-Driven Latent Generation Method for Scientific Data
• Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Images
• Efficient Neural Representation of Volumetric Data using Coordinate-Based Networks
• Deep Learning-Based Feature-Aware Data Modeling for Complex Physics Simulations