Learning disentangled representation for classical models
arXiv:2110.08082 · doi:10.1103/PhysRevB.105.245149
Abstract
Finding disentangled representation plays a predominant role in the success of modern deep learning applications, but the results lack a straightforward explanation. Here we apply the information bottleneck method and its -VAE implementation to find the disentangled low-dimensional representation of classical models. For the Ising model, our results reveal a deep connection between the disentangled features and the physical order parameters, and the widely-used Bernoulli decoder is found to be learning a mean-field Hamiltonian at fixed temperature. This analogy motivates us to extend the application of -VAE to more complex classical models with non-binary variables using different decoder neural network and propose a modified architecture -VAE to enforce thermal fluctuations in generated samples. Our work provides a way to design novel physics-informed algorithm that can yield learned features in potential correspondence with real physical properties.
arXiv comment10 pages, 7 figures, 2 tables
References in corpus (29)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Auto-Encoding Variational Bayes
- TensorFlow: A system for large-scale machine learning
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- The information bottleneck method
- Solving the Quantum Many-Body Problem with Artificial Neural Networks
- Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
- Many-body quantum state tomography with neural networks
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Disentangling by Factorising
- Discovering Phase Transitions with Unsupervised Learning
- Quantum generative adversarial learning
- Quantum Generative Adversarial Networks for Learning and Loading Random Distributions
- Quantum Boltzmann Machine
- Neural-Network Approach to Dissipative Quantum Many-Body Dynamics
- Learning Thermodynamics with Boltzmann Machines
- Variational Quantum Monte Carlo Method with a Neural-Network Ansatz for Open Quantum Systems
- Variational neural network ansatz for steady states in open quantum systems
- Mutual Information, Neural Networks and the Renormalization Group
- Constructing neural stationary states for open quantum many-body systems
- Neural Network Renormalization Group
- Latent Space Purification via Neural Density Operators
- Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow
- Deep Learning the Ising Model Near Criticality
- Super-resolving the Ising model with convolutional neural networks
- Smallest Neural Network to Learn the Ising Criticality
- Learning the Ising Model with Generative Neural Networks
- Hybridization fluctuations in the half-filled periodic Anderson model