A basic introduction to large deviations: Theory, applications, simulations

Abstract

The theory of large deviations deals with the probabilities of rare events (or fluctuations) that are exponentially small as a function of some parameter, e.g., the number of random components of a system, the time over which a stochastic system is observed, the amplitude of the noise perturbing a dynamical system or the temperature of a chemical reaction. The theory has applications in many different scientific fields, ranging from queuing theory to statistics and from finance to engineering. It is also increasingly used in statistical physics for studying both equilibrium and nonequilibrium systems. In this context, deep analogies can be made between familiar concepts of statistical physics, such as the entropy and the free energy, and concepts of large deviation theory having more technical names, such as the rate function and the scaled cumulant generating function. The first part of these notes introduces the basic elements of large deviation theory at a level appropriate for advanced undergraduate and graduate students in physics, engineering, chemistry, and mathematics. The focus there is on the simple but powerful ideas behind large deviation theory, stated in non-technical terms, and on the application of these ideas in simple stochastic processes, such as sums of independent and identically distributed random variables and Markov processes. Some physical applications of these processes are covered in exercises contained at the end of each section. In the second part, the problem of numerically evaluating large deviation probabilities is treated at a basic level. The fundamental idea of importance sampling is introduced there together with its sister idea, the exponential change of measure. Other numerical methods based on sample means and generating functions, with applications to Markov processes, are also covered.

58 pages, 6 figures; v2: minor typos corrected; v3: errors corrected and errata list added at the end of the paper; Modern Computational Science 11: Lecture Notes from the 3rd International Oldenburg Summer School, BIS-Verlag der Carl von Ossietzky Universitat Oldenburg, 2011

References in corpus (6)

• The large deviation approach to statistical mechanics
• Non equilibrium steady states: fluctuations and large deviations of the density and of the current
• Direct evaluation of large-deviation functions
• Stationary and Transient Work-Fluctuation Theorems for a Dragged Brownian Particle
• A numerical approach to large deviations in continuous-time
• DMRG-study of current and activity fluctuations near non-equilibrium phase transitions

Cited by in corpus (27)

• Introduction to dynamical large deviations of Markov processes
• Aspects of non-equilibrium in classical and quantum systems: slow relaxation and glasses, dynamical large deviations, quantum non-ergodicity, and open quantum dynamics
• Multiple-scale stochastic processes: decimation, averaging and beyond
• Nuw CDM cosmology from the weak lensing convergence PDF
• Dynamical phase transition in drifted Brownian motion
• Anomalous scaling of dynamical large deviations
• Game of cones: A nulling strategy for modelling lensing convergence in cones with large deviation theory
• Extreme events in transitional turbulence
• Multi-Layer Bilinear Generalized Approximate Message Passing
• The matter density PDF for modified gravity and dark energy with Large Deviations Theory
• Statistical Hypothesis Testing Based on Machine Learning: Large Deviations Analysis
• Inferring effective couplings with Restricted Boltzmann Machines
• An introduction to large deviations with applications in physics
• Full expectation value statistics for randomly sampled pure states in high-dimensional quantum systems
• Trajectory reweighting for non-equilibrium steady states
• Tensor-Network Approaches to Counting Statistics for the Current in a Boundary-Driven Diffusive System
• Numerical Aspects of Large Deviations
• Foundations of Stochastic Thermodynamics
• Efficient computation of extreme excursion probabilities for dynamical systems
• Coarse-graining via the fluctuation-dissipation theorem and large-deviation theory
• Risk aversion can promote cooperation
• Entropy as Noether charge for quasistatic gradient flow
• Estimation for High-Dimensional Multi-Layer Generalized Linear Model -- Part I: The Exact MMSE Estimator
• Towards neural reinforcement learning for large deviations in nonequilibrium systems with memory
• One-particle engine with a porous piston
• Replica Analysis for Generalized Linear Regression with IID Row Prior
• Permanent of random matrices from representation theory: moments, numerics, concentration, and comments on hardness of boson-sampling