68 citations · 493 across the 40 of their papers we have counts for
40 papers
Eight challenges in developing theory of intelligence
Haiping Huang
A good theory of mathematical beauty is more practical than any current observation, as new predictions of physical reality can be verified self-consistently. This belief applies t…
Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings
Wenxuan Zou, Haiping Huang
Dynamical mean-field theory is a powerful physics tool used to analyze the typical behavior of neural networks, where neurons can be recurrently connected, or multiple layers of ne…
Statistical mechanics of continual learning: variational principle and mean-field potential
Chan Li, Zhenye Huang, Wenxuan Zou +1
An obstacle to artificial general intelligence is set by continual learning of multiple tasks of different nature. Recently, various heuristic tricks, both from machine learning an…
Emergence of hierarchical modes from deep learning
Chan Li, Haiping Huang
Large-scale deep neural networks consume expensive training costs, but the training results in less-interpretable weight matrices constructing the networks. Here, we propose a mode…
Equivalence between algorithmic instability and transition to replica symmetry breaking in perceptron learning systems
Yang Zhao, Junbin Qiu, Mingshan Xie +1
Binary perceptron is a fundamental model of supervised learning for the non-convex optimization, which is a root of the popular deep learning. Binary perceptron is able to achieve…
Eigenvalue spectrum of neural networks with arbitrary Hebbian length
Jianwen Zhou, Zijian Jiang, Tianqi Hou +3
Associative memory is a fundamental function in the brain. Here, we generalize the standard associative memory model to include long-range Hebbian interactions at the learning stag…
Associative memory model with arbitrary Hebbian length
Zijian Jiang, Jianwen Zhou, Tianqi Hou +2
Conversion of temporal to spatial correlations in the cortex is one of the most intriguing functions in the brain. The learning at synapses triggering the correlation conversion ca…
Ensemble perspective for understanding temporal credit assignment
Wenxuan Zou, Chan Li, Haiping Huang
Recurrent neural networks are widely used for modeling spatio-temporal sequences in both nature language processing and neural population dynamics. However, understanding the tempo…
Data-driven effective model shows a liquid-like deep learning
Wenxuan Zou, Haiping Huang
The geometric structure of an optimization landscape is argued to be fundamentally important to support the success of deep neural network learning. A direct computation of the lan…
Relationship between manifold smoothness and adversarial vulnerability in deep learning with local errors
Zijian Jiang, Jianwen Zhou, Haiping Huang
Artificial neural networks can achieve impressive performances, and even outperform humans in some specific tasks. Nevertheless, unlike biological brains, the artificial neural net…
Weakly-correlated synapses promote dimension reduction in deep neural networks
Jianwen Zhou, Haiping Huang
By controlling synaptic and neural correlations, deep learning has achieved empirical successes in improving classification performances. How synaptic correlations affect neural co…
Learning credit assignment
Chan Li, Haiping Huang
Deep learning has achieved impressive prediction accuracies in a variety of scientific and industrial domains. However, the nested non-linear feature of deep learning makes the lea…
Variational mean-field theory for training restricted Boltzmann machines with binary synapses
Haiping Huang
Unsupervised learning requiring only raw data is not only a fundamental function of the cerebral cortex, but also a foundation for a next generation of artificial neural networks.…
Statistical physics of unsupervised learning with prior knowledge in neural networks
Tianqi Hou, Haiping Huang
Integrating sensory inputs with prior beliefs from past experiences in unsupervised learning is a common and fundamental characteristic of brain or artificial neural computation. H…
Minimal model of permutation symmetry in unsupervised learning
Tianqi Hou, K. Y. Michael Wong, Haiping Huang
Permutation of any two hidden units yields invariant properties in typical deep generative neural networks. This permutation symmetry plays an important role in understanding the c…
Mechanisms of dimensionality reduction and decorrelation in deep neural networks
Haiping Huang
Deep neural networks are widely used in various domains. However, the nature of computations at each layer of the deep networks is far from being well understood. Increasing the in…
Random active path model of deep neural networks with diluted binary synapses
Haiping Huang, Alireza Goudarzi
Deep learning has become a powerful and popular tool for a variety of machine learning tasks. However, it is challenging to understand the mechanism of deep learning from a theoret…
Role of zero synapses in unsupervised feature learning
Haiping Huang
Synapses in real neural circuits can take discrete values, including zero (silent or potential) synapses. The computational role of zero synapses in unsupervised feature learning o…
Reinforced stochastic gradient descent for deep neural network learning
Haiping Huang, Taro Toyoizumi
Stochastic gradient descent (SGD) is a standard optimization method to minimize a training error with respect to network parameters in modern neural network learning. However, it t…
Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapses
Haiping Huang
Revealing hidden features in unlabeled data is called unsupervised feature learning, which plays an important role in pretraining a deep neural network. Here we provide a statistic…
Unsupervised feature learning from finite data by message passing: discontinuous versus continuous phase transition
Haiping Huang, Taro Toyoizumi
Unsupervised neural network learning extracts hidden features from unlabeled training data. This is used as a pretraining step for further supervised learning in deep networks. Hen…
Clustering of neural codewords revealed by a first-order phase transition
Haiping Huang, Taro Toyoizumi
A network of neurons in the central nervous system collectively represents information by its spiking activity states. Typically observed states, i.e., codewords, occupy only a lim…
Theory of population coupling and applications to describe high order correlations in large populations of interacting neurons
Haiping Huang
To understand the collective spiking activity in neuronal populations, it is essential to reveal basic circuit variables responsible for these emergent functional states. Here, I d…
Effects of hidden nodes on network structure inference
Haiping Huang
Effects of hidden nodes on inference quality of observed network structure are explored based on a disordered Ising model with hidden nodes. We first study analytically small syste…