Network Inference with Hidden Units

Abstract

We derive learning rules for finding the connections between units in stochastic dynamical networks from the recorded history of a ``visible'' subset of the units. We consider two models. In both of them, the visible units are binary and stochastic. In one model the ``hidden'' units are continuous-valued, with sigmoidal activation functions, and in the other they are binary and stochastic like the visible ones. We derive exact learning rules for both cases. For the stochastic case, performing the exact calculation requires, in general, repeated summations over an number of configurations that grows exponentially with the size of the system and the data length, which is not feasible for large systems. We derive a mean field theory, based on a factorized ansatz for the distribution of hidden-unit states, which offers an attractive alternative for large systems. We present the results of some numerical calculations that illustrate key features of the two models and, for the stochastic case, the exact and approximate calculations.

References in corpus (3)

• The Ising Model for Neural Data: Model Quality and Approximate Methods for Extracting Functional Connectivity
• Mean Field Theory For Non-Equilibrium Network Reconstruction
• Learning and inference in a nonequilibrium Ising model with hidden nodes

Cited by in corpus (14)

• Distributions of covariances as a window into the operational regime of neuronal networks
• Two types of criticality in the brain
• Predicting how and when hidden neurons skew measured synaptic interactions
• Learning with hidden variables
• Inverse Ising problem in continuous time: A latent variable approach
• Belief-Propagation and replicas for inference and learning in a kinetic Ising model with hidden spins
• Inferring hidden states in a random kinetic Ising model: replica analysis
• Data-driven inference of hidden nodes in networks
• Effects of hidden nodes on network structure inference
• Inference for dynamics of continuous variables: the Extended Plefka Expansion with hidden nodes
• Learning of couplings for random asymmetric kinetic Ising models revisited: random correlation matrices and learning curves
• The appropriateness of ignorance in the inverse kinetic Ising model
• Inference of stochastic time series with missing data
• Nonequilibrium Green's functions for functional connectivity in the brain