Training neural networks under physical constraints using a stochastic augmented Lagrangian approach

Abstract

We investigate the physics-constrained training of an encoder-decoder neural network for approximating the Fokker-Planck-Landau collision operator in the 5-dimensional kinetic fusion simulation in XGC. To train this network, we propose a stochastic augmented Lagrangian approach that utilizes pyTorch's native stochastic gradient descent method to solve the inner unconstrained minimization subproblem, paired with a heuristic update for the penalty factor and Lagrange multipliers in the outer augmented Lagrangian loop. Our training results for a single ion species case, with self-collisions and collision against electrons, show that the proposed stochastic augmented Lagrangian approach can achieve higher model prediction accuracy than training with a fixed penalty method for our application problem, with the accuracy high enough for practical applications in kinetic simulations.

References in corpus (4)

• Turbulence Modeling in the Age of Data
• Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems
• Deep Neural Networks for Data-Driven Turbulence Models
• Embedding Hard Physical Constraints in Neural Network Coarse-Graining of 3D Turbulence

Cited by in corpus (3)

• Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion
• Multi-Objective Physics-Guided Recurrent Neural Networks for Identifying Non-Autonomous Dynamical Systems
• Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modeling