Training robust neural networks using Lipschitz bounds

Abstract

Due to their susceptibility to adversarial perturbations, neural networks (NNs) are hardly used in safety-critical applications. One measure of robustness to such perturbations in the input is the Lipschitz constant of the input-output map defined by an NN. In this work, we propose a framework to train multi-layer NNs while at the same time encouraging robustness by keeping their Lipschitz constant small, thus addressing the robustness issue. More specifically, we design an optimization scheme based on the Alternating Direction Method of Multipliers that minimizes not only the training loss of an NN but also its Lipschitz constant resulting in a semidefinite programming based training procedure that promotes robustness. We design two versions of this training procedure. The first one includes a regularizer that penalizes an accurate upper bound on the Lipschitz constant. The second one allows to enforce a desired Lipschitz bound on the NN at all times during training. Finally, we provide two examples to show that the proposed framework successfully increases the robustness of NNs.

References in corpus (4)

• Explaining and Harnessing Adversarial Examples
• Natural Language Processing (almost) from Scratch
• Lipschitz constant estimation of Neural Networks via sparse polynomial optimization
• Exactly Computing the Local Lipschitz Constant of ReLU Networks

Cited by in corpus (23)

• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• Learning Security Classifiers with Verified Global Robustness Properties
• Reinforcement Learning for Distributed Transient Frequency Control with Stability and Safety Guarantees
• Training robust and generalizable quantum models
• ShieldNN: A Provably Safe NN Filter for Unsafe NN Controllers
• Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers
• System Identification Through Lipschitz Regularized Deep Neural Networks
• Quantum computing through the lens of control: A tutorial introduction
• Convergence of Deep Fictitious Play for Stochastic Differential Games
• SafEDMD: A Koopman-based data-driven controller design framework for nonlinear dynamical systems
• Synthesis of Stabilizing Recurrent Equilibrium Network Controllers
• Lipschitz Bounds and Provably Robust Training by Laplacian Smoothing
• Combining Robust Control and Machine Learning for Uncertain Nonlinear Systems Subject to Persistent Disturbances
• Provably Correct Training of Neural Network Controllers Using Reachability Analysis
• Imitation Learning of MPC with Neural Networks: Error Guarantees and Sparsification
• Learning Stable and Robust Linear Parameter-Varying State-Space Models
• Stabilizing Multimodal Autoencoders: A Theoretical and Empirical Analysis of Fusion Strategies
• A mathematical certification for positivity conditions in Neural Networks with applications to partial monotonicity and Trustworthy AI
• LipKernel: Lipschitz-Bounded Convolutional Neural Networks via Dissipative Layers
• Robust Control Design and Analysis Based on Lifting Linearization of Nonlinear Systems Under Uncertain Initial Conditions
• Norm-induced Cuts: Outer Approximation for Lipschitzian Constraint Functions
• Adversarial flows: A gradient flow characterization of adversarial attacks
• Constrained Performance Boosting Control for Nonlinear Systems