Volumetric Isosurface Rendering with Deep Learning-Based Super-Resolution

Abstract

Rendering an accurate image of an isosurface in a volumetric field typically requires large numbers of data samples. Reducing the number of required samples lies at the core of research in volume rendering. With the advent of deep learning networks, a number of architectures have been proposed recently to infer missing samples in multi-dimensional fields, for applications such as image super-resolution and scan completion. In this paper, we investigate the use of such architectures for learning the upscaling of a low-resolution sampling of an isosurface to a higher resolution, with high fidelity reconstruction of spatial detail and shading. We introduce a fully convolutional neural network, to learn a latent representation generating a smooth, edge-aware normal field and ambient occlusions from a low-resolution normal and depth field. By adding a frame-to-frame motion loss into the learning stage, the upscaling can consider temporal variations and achieves improved frame-to-frame coherence. We demonstrate the quality of the network for isosurfaces which were never seen during training, and discuss remote and in-situ visualization as well as focus+context visualization as potential applications

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations
• A Generative Model for Volume Rendering
• Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks

Cited by in corpus (11)

• A comparative study of convolutional neural network models for wind field downscaling
• Fast Neural Representations for Direct Volume Rendering
• Differentiable Direct Volume Rendering
• Learning Adaptive Sampling and Reconstruction for Volume Visualization
• Deep Volumetric Ambient Occlusion
• Deep Hierarchical Super Resolution for Scientific Data
• Exploratory Lagrangian-Based Particle Tracing Using Deep Learning
• Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Images
• Efficient Neural Representation of Volumetric Data using Coordinate-Based Networks
• Monocular Depth Decomposition of Semi-Transparent Volume Renderings
• Interactive volume illumination of slice-based ray casting