Fisher information flow in artificial neural networks

Abstract

The estimation of continuous parameters from measured data plays a central role in many fields of physics. A key tool in understanding and improving such estimation processes is the concept of Fisher information, which quantifies how information about unknown parameters propagates through a physical system and determines the ultimate limits of precision. With Artificial Neural Networks (ANNs) gradually becoming an integral part of many measurement systems, it is essential to understand how they process and transmit parameter-relevant information internally. Here, we present a method to monitor the flow of Fisher information through an ANN performing a parameter estimation task, tracking it from the input to the output layer. We show that optimal estimation performance corresponds to the maximal transmission of Fisher information, and that training beyond this point results in information loss due to overfitting. This provides a model-free stopping criterion for network training-eliminating the need for a separate validation dataset. To demonstrate the practical relevance of our approach, we apply it to a network trained on data from an imaging experiment, highlighting its effectiveness in a realistic physical setting.

17 pages, 12 figures

References in corpus (20)

• Overcoming catastrophic forgetting in neural networks
• Advances in Quantum Metrology
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems
• Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media
• Levitodynamics: Levitation and control of microscopic objects in vacuum
• Automated Search for new Quantum Experiments
• Quantum inference of states and processes
• Graph Neural Networks in Particle Physics
• Learning Quantum Systems
• Generalized massive optimal data compression
• Maximum information states for coherent scattering measurements
• Determination of the edge of criticality in echo state networks through Fisher information maximization
• A Lower Bound for the Fisher Information Measure
• Empirical non-parametric estimation of the Fisher Information
• Continuity Equation for the Flow of Fisher Information in Wave Scattering
• A Pessimistic Approximation for the Fisher Information Measure
• Optimal control of coherent light scattering for binary decision problems
• Non-parametric estimation of Fisher information from real data