Learning disentangled representation for classical models

Abstract

Finding disentangled representation plays a predominant role in the success of modern deep learning applications, but the results lack a straightforward explanation. Here we apply the information bottleneck method and its $\beta$-VAE implementation to find the disentangled low-dimensional representation of classical models. For the Ising model, our results reveal a deep connection between the disentangled features and the physical order parameters, and the widely-used Bernoulli decoder is found to be learning a mean-field Hamiltonian at fixed temperature. This analogy motivates us to extend the application of $\beta$-VAE to more complex classical models with non-binary variables using different decoder neural network and propose a modified architecture ${\beta }^2$-VAE to enforce thermal fluctuations in generated samples. Our work provides a way to design novel physics-informed algorithm that can yield learned features in potential correspondence with real physical properties.

10 pages, 7 figures, 2 tables

References in corpus (13)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Quantum generative adversarial learning
• Quantum Generative Adversarial Networks for Learning and Loading Random Distributions
• Neural-Network Approach to Dissipative Quantum Many-Body Dynamics
• Variational Quantum Monte Carlo Method with a Neural-Network Ansatz for Open Quantum Systems
• Variational neural network ansatz for steady states in open quantum systems
• Constructing neural stationary states for open quantum many-body systems
• Super-resolving the Ising model with convolutional neural networks
• Smallest Neural Network to Learn the Ising Criticality
• Learning the Ising Model with Generative Neural Networks
• Hybridization fluctuations in the half-filled periodic Anderson model