355 citations · 1.6k across the 54 of their papers we have counts for
7 papers · 1 filter
Biologically-plausible learning algorithms can scale to large datasets
Will Xiao, Honglin Chen, Qianli Liao +1
The backpropagation (BP) algorithm is often thought to be biologically implausible in the brain. One of the main reasons is that BP requires symmetric weight matrices in the feedfo…
A Surprising Linear Relationship Predicts Test Performance in Deep Networks
Qianli Liao, Brando Miranda, Andrzej Banburski +2
Given two networks with the same training loss on a dataset, when would they have drastically different test losses and errors? Better understanding of this question of generalizat…
Theory IIIb: Generalization in Deep Networks
Tomaso Poggio, Qianli Liao, Brando Miranda +3
A main puzzle of deep neural networks (DNNs) revolves around the apparent absence of "overfitting", defined in this paper as follows: the expected error does not get worse when inc…
Approximate inference with Wasserstein gradient flows
Charlie Frogner, Tomaso Poggio
We present a novel approximate inference method for diffusion processes, based on the Wasserstein gradient flow formulation of the diffusion. In this formulation, the time-dependen…
An analysis of training and generalization errors in shallow and deep networks
Hrushikesh Mhaskar, Tomaso Poggio
This paper is motivated by an open problem around deep networks, namely, the apparent absence of over-fitting despite large over-parametrization which allows perfect fitting of the…
Theory of Deep Learning III: explaining the non-overfitting puzzle
Tomaso Poggio, Kenji Kawaguchi, Qianli Liao +5
A main puzzle of deep networks revolves around the absence of overfitting despite large overparametrization and despite the large capacity demonstrated by zero training error on ra…
Theory of Deep Learning IIb: Optimization Properties of SGD
Chiyuan Zhang, Qianli Liao, Alexander Rakhlin +3
In Theory IIb we characterize with a mix of theory and experiments the optimization of deep convolutional networks by Stochastic Gradient Descent. The main new result in this paper…