355 citations · 1.6k across the 54 of their papers we have counts for
55 papers
System identification of neural systems: If we got it right, would we know?
Yena Han, Tomaso Poggio, Brian Cheung
Artificial neural networks are being proposed as models of parts of the brain. The networks are compared to recordings of biological neurons, and good performance in reproducing ne…
Norm-based Generalization Bounds for Compositionally Sparse Neural Networks
Tomer Galanti, Mengjia Xu, Liane Galanti +1
In this paper, we investigate the Rademacher complexity of deep sparse neural networks, where each neuron receives a small number of inputs. We prove generalization bounds for mult…
Iterative regularization in classification via hinge loss diagonal descent
Vassilis Apidopoulos, Tomaso Poggio, Lorenzo Rosasco +1
Iterative regularization is a classic idea in regularization theory, that has recently become popular in machine learning. On the one hand, it allows to design efficient algorithms…
SGD and Weight Decay Secretly Minimize the Rank of Your Neural Network
Tomer Galanti, Zachary S. Siegel, Aparna Gupte +1
We investigate the inherent bias of Stochastic Gradient Descent (SGD) toward learning low-rank weight matrices during the training of deep neural networks. Our results demonstrate…
Neural-guided, Bidirectional Program Search for Abstraction and Reasoning
Simon Alford, Anshula Gandhi, Akshay Rangamani +6
One of the challenges facing artificial intelligence research today is designing systems capable of utilizing systematic reasoning to generalize to new tasks. The Abstraction and R…
Distribution of Classification Margins: Are All Data Equal?
Andrzej Banburski, Fernanda De La Torre, Nishka Pant +2
Recent theoretical results show that gradient descent on deep neural networks under exponential loss functions locally maximizes classification margin, which is equivalent to minim…
The Effects of Image Distribution and Task on Adversarial Robustness
Owen Kunhardt, Arturo Deza, Tomaso Poggio
In this paper, we propose an adaptation to the area under the curve (AUC) metric to measure the adversarial robustness of a model over a particular -interval (inter…
Explicit regularization and implicit bias in deep network classifiers trained with the square loss
Tomaso Poggio, Qianli Liao
Deep ReLU networks trained with the square loss have been observed to perform well in classification tasks. We provide here a theoretical justification based on analysis of the ass…
CUDA-Optimized real-time rendering of a Foveated Visual System
Elian Malkin, Arturo Deza, Tomaso Poggio
The spatially-varying field of the human visual system has recently received a resurgence of interest with the development of virtual reality (VR) and neural networks. The computat…
Biologically Inspired Mechanisms for Adversarial Robustness
Manish V. Reddy, Andrzej Banburski, Nishka Pant +1
A convolutional neural network strongly robust to adversarial perturbations at reasonable computational and performance cost has not yet been demonstrated. The primate visual ventr…
For interpolating kernel machines, minimizing the norm of the ERM solution minimizes stability
Akshay Rangamani, Lorenzo Rosasco, Tomaso Poggio
We study the average $\mbox{CV}_{loo}$ stability of kernel ridge-less regression and derive corresponding risk bounds. We show that the interpolating solution with minimum norm min…
Hierarchically Compositional Tasks and Deep Convolutional Networks
Arturo Deza, Qianli Liao, Andrzej Banburski +1
The main success stories of deep learning, starting with ImageNet, depend on deep convolutional networks, which on certain tasks perform significantly better than traditional shall…
Double descent in the condition number
Tomaso Poggio, Gil Kur, Andrzej Banburski
In solving a system of linear equations in variables , the condition number of the matrix measures how much errors in the data affect the solution .…
Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization
Tomaso Poggio, Andrzej Banburski, Qianli Liao
While deep learning is successful in a number of applications, it is not yet well understood theoretically. A satisfactory theoretical characterization of deep learning however, is…
Deep Recurrent Architectures for Seismic Tomography
Amir Adler, Mauricio Araya-Polo, Tomaso Poggio
This paper introduces novel deep recurrent neural network architectures for Velocity Model Building (VMB), which is beyond what Araya-Polo et al 2018 pioneered with the Machine Lea…
Function approximation by deep networks
H. N. Mhaskar, T. Poggio
We show that deep networks are better than shallow networks at approximating functions that can be expressed as a composition of functions described by a directed acyclic graph, be…
Theory III: Dynamics and Generalization in Deep Networks
Andrzej Banburski, Qianli Liao, Brando Miranda +4
The key to generalization is controlling the complexity of the network. However, there is no obvious control of complexity -- such as an explicit regularization term -- in the trai…
Biologically-plausible learning algorithms can scale to large datasets
Will Xiao, Honglin Chen, Qianli Liao +1
The backpropagation (BP) algorithm is often thought to be biologically implausible in the brain. One of the main reasons is that BP requires symmetric weight matrices in the feedfo…
A Surprising Linear Relationship Predicts Test Performance in Deep Networks
Qianli Liao, Brando Miranda, Andrzej Banburski +2
Given two networks with the same training loss on a dataset, when would they have drastically different test losses and errors? Better understanding of this question of generalizat…
Theory IIIb: Generalization in Deep Networks
Tomaso Poggio, Qianli Liao, Brando Miranda +3
A main puzzle of deep neural networks (DNNs) revolves around the apparent absence of "overfitting", defined in this paper as follows: the expected error does not get worse when inc…
Approximate inference with Wasserstein gradient flows
Charlie Frogner, Tomaso Poggio
We present a novel approximate inference method for diffusion processes, based on the Wasserstein gradient flow formulation of the diffusion. In this formulation, the time-dependen…
An analysis of training and generalization errors in shallow and deep networks
Hrushikesh Mhaskar, Tomaso Poggio
This paper is motivated by an open problem around deep networks, namely, the apparent absence of over-fitting despite large over-parametrization which allows perfect fitting of the…
Theory of Deep Learning III: explaining the non-overfitting puzzle
Tomaso Poggio, Kenji Kawaguchi, Qianli Liao +5
A main puzzle of deep networks revolves around the absence of overfitting despite large overparametrization and despite the large capacity demonstrated by zero training error on ra…
Theory of Deep Learning IIb: Optimization Properties of SGD
Chiyuan Zhang, Qianli Liao, Alexander Rakhlin +3
In Theory IIb we characterize with a mix of theory and experiments the optimization of deep convolutional networks by Stochastic Gradient Descent. The main new result in this paper…