Controlling Neural Level Sets

Abstract

The level sets of neural networks represent fundamental properties such as decision boundaries of classifiers and are used to model non-linear manifold data such as curves and surfaces. Thus, methods for controlling the neural level sets could find many applications in machine learning. In this paper we present a simple and scalable approach to directly control level sets of a deep neural network. Our method consists of two parts: (i) sampling of the neural level sets, and (ii) relating the samples' positions to the network parameters. The latter is achieved by a sample network that is constructed by adding a single fixed linear layer to the original network. In turn, the sample network can be used to incorporate the level set samples into a loss function of interest. We have tested our method on three different learning tasks: improving generalization to unseen data, training networks robust to adversarial attacks, and curve and surface reconstruction from point clouds. For surface reconstruction, we produce high fidelity surfaces directly from raw 3D point clouds. When training small to medium networks to be robust to adversarial attacks we obtain robust accuracy comparable to state-of-the-art methods.

arXiv commentNeurIPS 2019

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Theoretically Principled Trade-off between Robustness and Accuracy
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Robust Large Margin Deep Neural Networks
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• Understanding Deep Neural Networks with Rectified Linear Units
• Learning Implicit Fields for Generative Shape Modeling
• On the Depth of Deep Neural Networks: A Theoretical View

Cited by in corpus (14)

• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance
• Implicit Geometric Regularization for Learning Shapes
• Adversarial Robustness via Fisher-Rao Regularization
• Learning Occupancy Function from Point Clouds for Surface Reconstruction
• FroDO: From Detections to 3D Objects
• NeuSample: Neural Sample Field for Efficient View Synthesis
• Bag of Tricks for Adversarial Training
• Playing it Safe: Adversarial Robustness with an Abstain Option
• Neural Fields in Visual Computing and Beyond
• MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis
• Signed Distance Function Computation from an Implicit Surface
• Enhance Diffusion to Improve Robust Generalization
• Learning Barrier Functions with Memory for Robust Safe Navigation