Semiparametric energy-based probabilistic models
arXiv:1605.07371
Abstract
Probabilistic models can be defined by an energy function, where the probability of each state is proportional to the exponential of the state's negative energy. This paper considers a generalization of energy-based models in which the probability of a state is proportional to an arbitrary positive, strictly decreasing, and twice differentiable function of the state's energy. The precise shape of the nonlinear map from energies to unnormalized probabilities has to be learned from data together with the parameters of the energy function. As a case study we show that the above generalization of a fully visible Boltzmann machine yields an accurate model of neural activity of retinal ganglion cells. We attribute this success to the model's ability to easily capture distributions whose probabilities span a large dynamic range, a possible consequence of latent variables that globally couple the system. Similar features have recently been observed in many datasets, suggesting that our new method has wide applicability.
8 pages, 3 figures
References in corpus (11)
- Weak pairwise correlations imply strongly correlated network states in a neural population
- Are biological systems poised at criticality?
- Maximum entropy models for antibody diversity
- Searching for collective behavior in a network of real neurons
- Fast Algorithm to Calculate Density of States
- Natural images from the birthplace of the human eye
- Zipf's law and criticality in multivariate data without fine-tuning
- When do generalized entropies apply? How phase space volume determines entropy
- Dynamical criticality in the collective activity of a population of retinal neurons
- Thermodynamics of natural images
- Zipf's law arises naturally in structured, high-dimensional data