Information Field Theory and Artificial Intelligence

Abstract

Information field theory (IFT), the information theory for fields, is a mathematical framework for signal reconstruction and non-parametric inverse problems. Artificial intelligence (AI) and machine learning (ML) aim at generating intelligent systems including such for perception, cognition, and learning. This overlaps with IFT, which is designed to address perception, reasoning, and inference tasks. Here, the relation between concepts and tools in IFT and those in AI and ML research are discussed. In the context of IFT, fields denote physical quantities that change continuously as a function of space (and time) and information theory refers to Bayesian probabilistic logic equipped with the associated entropic information measures. Reconstructing a signal with IFT is a computational problem similar to training a generative neural network (GNN) in ML. In this paper, the process of inference in IFT is reformulated in terms of GNN training. In contrast to classical neural networks, IFT based GNNs can operate without pre-training thanks to incorporating expert knowledge into their architecture. Furthermore, the cross-fertilization of variational inference methods used in IFT and ML are discussed. These discussions suggests that IFT is well suited to address many problems in AI and ML research and application.

arXiv comment12 pages, three figures, invited talk at MaxEnt2020/2021, reviewed and published by Entropy

References in corpus (34)

• Variational Inference with Normalizing Flows
• Deep Image Prior
• An improved map of the Galactic Faraday sky
• Estimating extragalactic Faraday rotation
• Information field theory for cosmological perturbation reconstruction and non-linear signal analysis
• Resolving nearby dust clouds
• NIFTY - Numerical Information Field Theory - a versatile Python library for signal inference
• The Denoised, Deconvolved, and Decomposed Fermi $\gamma$-ray sky - An application of the D${}^3$PO algorithm
• Reconstruction of signals with unknown spectra in information field theory with parameter uncertainty
• Charting nearby dust clouds using Gaia data only
• The Galactic Faraday depth sky revisited
• Inference with minimal Gibbs free energy in information field theory
• Information theory for fields
• Comparison of classical and Bayesian imaging in radio interferometry
• Geometric variational inference
• Unified Radio Interferometric Calibration and Imaging with Joint Uncertainty Quantification
• Galactic dark matter search via phenomenological astrophysics modeling
• A new approach to multi-frequency synthesis in radio interferometry
• Optimal Belief Approximation
• Information field dynamics for simulation scheme construction
• Bayesian inference of three-dimensional gas maps: I. Galactic CO
• Improving stochastic estimates with inference methods: calculating matrix diagonals
• Reconstructing non-repeating radio pulses with Information Field Theory
• Denoising, deconvolving and decomposing multi-domain photon observations- The D4PO algorithm
• A Bayesian method for pulsar template generation
• Field dynamics inference for local and causal interactions
• A New Method for Analysing Dark Matter Direct Detection Data
• Field dynamics inference via spectral density estimation
• Operator Calculus for Information Field Theory
• Towards information optimal simulation of partial differential equations
• A Bayesian Model for Bivariate Causal Inference
• All-sky reconstruction of the primordial scalar potential from WMAP temperature data
• Bayesian Reasoning with Trained Neural Networks
• Separating diffuse from point-like sources - a Bayesian approach

Cited by in corpus (4)

• Physics-informed Information Field Theory for Modeling Physical Systems with Uncertainty Quantification
• First spatio-spectral Bayesian imaging of SN1006 in X-ray
• Non-parametric Bayesian reconstruction of Galactic magnetic fields using Information Field Theory: The inclusion of line-of-sight information in ultra-high energy cosmic ray backtracking
• Inferring Evidence from Nested Sampling Data via Information Field Theory