Differentiable Surface Splatting for Point-based Geometry Processing

Abstract

We propose Differentiable Surface Splatting (DSS), a high-fidelity differentiable renderer for point clouds. Gradients for point locations and normals are carefully designed to handle discontinuities of the rendering function. Regularization terms are introduced to ensure uniform distribution of the points on the underlying surface. We demonstrate applications of DSS to inverse rendering for geometry synthesis and denoising, where large scale topological changes, as well as small scale detail modifications, are accurately and robustly handled without requiring explicit connectivity, outperforming state-of-the-art techniques. The data and code are at https://github.com/yifita/DSS.

This version is contains camera-ready manuscript for SIGGRAPH Asia 2019

References in corpus (6)

• PCPNET: Learning Local Shape Properties from Raw Point Clouds
• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
• Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
• RenderNet: A deep convolutional network for differentiable rendering from 3D shapes
• 3D-Aware Scene Manipulation via Inverse Graphics
• Pix2Vex: Image-to-Geometry Reconstruction using a Smooth Differentiable Renderer

Cited by in corpus (35)

• 2D Gaussian Splatting for Geometrically Accurate Radiance Fields
• 3D Gaussian as a New Era: A Survey
• Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance
• Differentiable Rendering: A Survey
• Accelerating 3D Deep Learning with PyTorch3D
• Neural Point Catacaustics for Novel-View Synthesis of Reflections
• Orienting Point Clouds with Dipole Propagation
• Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data
• Lagrangian Neural Style Transfer for Fluids
• CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians
• Artemis: Articulated Neural Pets with Appearance and Motion synthesis
• VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Points
• ADOP: Approximate Differentiable One-Pixel Point Rendering
• Multiview Textured Mesh Recovery by Differentiable Rendering
• MG-SLAM: Structure Gaussian Splatting SLAM with Manhattan World Hypothesis
• SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
• Deep Implicit Moving Least-Squares Functions for 3D Reconstruction
• Non-Local Part-Aware Point Cloud Denoising
• Weakly Supervised Learning of Multi-Object 3D Scene Decompositions Using Deep Shape Priors
• Image-GS: Content-Adaptive Image Representation via 2D Gaussians
• Style-based Point Generator with Adversarial Rendering for Point Cloud Completion
• A Neural Rendering Framework for Free-Viewpoint Relighting
• When Gaussian Meets Surfel: Ultra-fast High-fidelity Radiance Field Rendering
• End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds
• SpecGaussian with Latent Features: A High-quality Modeling of the View-dependent Appearance for 3D Gaussian Splatting
• Deep 3D-to-2D Watermarking: Embedding Messages in 3D Meshes and Extracting Them from 2D Renderings
• Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations
• SweepEvGS: Event-Based 3D Gaussian Splatting for Macro and Micro Radiance Field Rendering from a Single Sweep
• SeqXY2SeqZ: Structure Learning for 3D Shapes by Sequentially Predicting 1D Occupancy Segments From 2D Coordinates
• Differentiable Surface Rendering via Non-Differentiable Sampling
• Rethinking Point Cloud Filtering: A Non-Local Position Based Approach
• FC2T2: The Fast Continuous Convolutional Taylor Transform with Applications in Vision and Graphics
• GaussianFocus: Constrained Attention Focus for 3D Gaussian Splatting
• Point-Based Neural Rendering with Per-View Optimization
• Incomplete Gamma Kernels: Generalizing Locally Optimal Projection Operators