Flexible Isosurface Extraction for Gradient-Based Mesh Optimization

Abstract

This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative modeling, and inverse physics. Existing implementations adapt classic isosurface extraction algorithms like Marching Cubes or Dual Contouring; these techniques were designed to extract meshes from fixed, known fields, and in the optimization setting they lack the degrees of freedom to represent high-quality feature-preserving meshes, or suffer from numerical instabilities. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives. Our main insight is to introduce additional carefully-chosen parameters into the representation, which allow local flexible adjustments to the extracted mesh geometry and connectivity. These parameters are updated along with the underlying scalar field via automatic differentiation when optimizing for a downstream task. We base our extraction scheme on Dual Marching Cubes for improved topological properties, and present extensions to optionally generate tetrahedral and hierarchically-adaptive meshes. Extensive experiments validate FlexiCubes on both synthetic benchmarks and real-world applications, showing that it offers significant improvements in mesh quality and geometric fidelity.

SIGGRAPH 2023. Project page: https://research.nvidia.com/labs/toronto-ai/flexicubes/

References in corpus (12)

• Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer
• Point2Mesh: A Self-Prior for Deformable Meshes
• GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images
• StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
• Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning
• Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis
• NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
• VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids
• Learning Deformable Tetrahedral Meshes for 3D Reconstruction
• Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising
• Appearance-Driven Automatic 3D Model Simplification
• A Level Set Theory for Neural Implicit Evolution under Explicit Flows

Cited by in corpus (15)

• Differential Walk on Spheres
• SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes
• TetWeave: Isosurface Extraction using On-The-Fly Delaunay Tetrahedral Grids for Gradient-Based Mesh Optimization
• Neurally Integrated Finite Elements for Differentiable Elasticity on Evolving Domains
• LFS-Aware Surface Reconstruction from Unoriented 3D Point Clouds
• Automatic Aorta Segmentation with Heavily Augmented, High-Resolution 3-D ResUNet: Contribution to the SEG.A Challenge
• ConsistentDreamer: View-Consistent Meshes Through Balanced Multi-View Gaussian Optimization
• McGrids: Monte Carlo-Driven Adaptive Grids for Iso-Surface Extraction
• PaMO: Parallel Mesh Optimization for Intersection-Free Low-Poly Modeling on the GPU
• DeMapGS: Simultaneous Mesh Deformation and Surface Attribute Mapping via Gaussian Splatting
• PhysID: Physics-based Interactive Dynamics from a Single-view Image
• DR-GS: Physically-Based Deformable and Relightable 2D Gaussians
• Differentiable Voxelization of Surface Representations
• LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation
• MTFusion: Reconstructing Any 3D Object from Single Image Using Multi-word Textual Inversion