A Neural Network Perturbation Theory Based on the Born Series

Abstract

Deep Learning using the eponymous deep neural networks (DNNs) has become an attractive approach towards various data-based problems of theoretical physics in the past decade. There has been a clear trend to deeper architectures containing increasingly more powerful and involved layers. Contrarily, Taylor coefficients of DNNs still appear mainly in the light of interpretability studies, where they are computed at most to first order. However, especially in theoretical physics numerous problems benefit from accessing higher orders, as well. This gap motivates a general formulation of neural network (NN) Taylor expansions. Restricting our analysis to multilayer perceptrons (MLPs) and introducing quantities we refer to as propagators and vertices, both depending on the MLP's weights and biases, we establish a graph-theoretical approach. Similarly to Feynman rules in quantum field theories, we can systematically assign diagrams containing propagators and vertices to the corresponding partial derivative. Examining this approach for S-wave scattering lengths of shallow potentials, we observe NNs to adapt their derivatives mainly to the leading order of the target function's Taylor expansion. To circumvent this problem, we propose an iterative NN perturbation theory. During each iteration we eliminate the leading order, such that the next-to-leading order can be faithfully learned during the subsequent iteration. After performing two iterations, we find that the first- and second-order Born terms are correctly adapted during the respective iterations. Finally, we combine both results to find a proxy that acts as a machine-learned second-order Born approximation.

arXiv comment29 pages, 4 figures, revised with more focus on neural network perturbation theory, accepted for publication in Phys. Rev. Research

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Decoupled Weight Decay Regularization
• Gaussian Error Linear Units (GELUs)
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Searching for Exotic Particles in High-Energy Physics with Deep Learning
• Jet Substructure at the Large Hadron Collider: A Review of Recent Advances in Theory and Machine Learning
• On Machine-Learned Classification of Variable Stars with Sparse and Noisy Time-Series Data
• Deep learning and the Schrödinger equation
• Constraining Effective Field Theories with Machine Learning
• An exact mapping between the Variational Renormalization Group and Deep Learning
• Machine Learning of Explicit Order Parameters: From the Ising Model to SU(2) Lattice Gauge Theory
• Predictions of nuclear $\beta$-decay half-lives with machine learning and their impacts on $r$ process
• SKYNET: an efficient and robust neural network training tool for machine learning in astronomy
• Methodology study of machine learning for the neutron star equation of state
• A machine learning study to identify spinodal clumping in high energy nuclear collisions
• Fast supersymmetry phenomenology at the Large Hadron Collider using machine learning techniques
• Visualizing Neural Network Developing Perturbation Theory

Cited by in corpus (3)

• Machine Learning in Nuclear Physics
• A Triumvirate of AI Driven Theoretical Discovery
• Morphology of three-body quantum states from machine learning