TherML: Thermodynamics of Machine Learning
arXiv:1807.04162
Abstract
In this work we offer a framework for reasoning about a wide class of existing objectives in machine learning. We develop a formal correspondence between this work and thermodynamics and discuss its implications.
arXiv commentPresented at the ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models
References in corpus (11)
- Auto-Encoding Variational Bayes
- The information bottleneck method
- Weight Uncertainty in Neural Networks
- Stochastic Gradient Hamiltonian Monte Carlo
- Gradient-based Hyperparameter Optimization through Reversible Learning
- Deep Variational Information Bottleneck
- The thermodynamics of prediction
- A Complete Recipe for Stochastic Gradient MCMC
- Multivariate Information Bottleneck
- A Bayesian Perspective on Generalization and Stochastic Gradient Descent
- Monte Carlo Methods for Rough Free Energy Landscapes: Population Annealing and Parallel Tempering
Cited by in corpus (13)
- Recent Advances in Autoencoder-Based Representation Learning
- State Predictive Information Bottleneck
- Dynamics-Aware Unsupervised Discovery of Skills
- Action and Perception as Divergence Minimization
- Learning about learning by many-body systems
- Differentiable Physics: A Position Piece
- Pareto-optimal clustering with the primal deterministic information bottleneck
- The Physics of Learning
- Dueling Decoders: Regularizing Variational Autoencoder Latent Spaces
- Likelihood Ratio Exponential Families
- Understanding Learning Dynamics of Binary Neural Networks via Information Bottleneck
- A Free-Energy Principle for Representation Learning
- Quantifying many-body learning far from equilibrium with representation learning