Gradient flows on graphons: existence, convergence, continuity equations
arXiv:2111.09459 · doi:10.1007/s10959-023-01271-8
Abstract
Wasserstein gradient flows on probability measures have found a host of applications in various optimization problems. They typically arise as the continuum limit of exchangeable particle systems evolving by some mean-field interaction involving a gradient-type potential. However, in many problems, such as in multi-layer neural networks, the so-called particles are edge weights on large graphs whose nodes are exchangeable. Such large graphs are known to converge to continuum limits called graphons as their size grow to infinity. We show that the Euclidean gradient flow of a suitable function of the edge-weights converges to a novel continuum limit given by a curve on the space of graphons that can be appropriately described as a gradient flow or, more technically, a curve of maximal slope. Several natural functions on graphons, such as homomorphism functions and the scalar entropy, are covered by our set-up, and the examples have been worked out in detail.
arXiv comment43+3 pages, 2 figures (Accepted version for publication in Journal of Theoretical Probability)
References in corpus (12)
- A Mean Field View of the Landscape of Two-Layers Neural Networks
- Graph limits and exchangeable random graphs
- Estimating and understanding exponential random graph models
- Sparse exchangeable graphs and their limits via graphon processes
- On replica symmetry of large deviations in random graphs
- On exchangeable random variables and the statistics of large graphs and hypergraphs
- A mean-field limit for certain deep neural networks
- Differential Calculus on Graphon Space
- On the asymptotics of constrained exponential random graphs
- Exchangeable random measures
- Graphons and cut metric on sigma-finite measure spaces
- On derivatives of graphon parameters