185 citations · 657 across the 21 of their papers we have counts for
21 papers
Hierarchical Reasoning Models: Perspectives and Misconceptions
Renee Ge, Qianli Liao, Tomaso Poggio
Transformers have demonstrated remarkable performance in natural language processing and related domains, as they largely focus on sequential, autoregressive next-token prediction…
How Biological Synapses Self-Assemble Gradient Learning
Qianli Liao, Liu Ziyin, Yulu Gan +4
Existing models of learning in the brain explain how given circuits learn, but not how biology could assemble those circuits in the first place. To address this gap, we formulate S…
Explicit regularization and implicit bias in deep network classifiers trained with the square loss
Tomaso Poggio, Qianli Liao
Deep ReLU networks trained with the square loss have been observed to perform well in classification tasks. We provide here a theoretical justification based on analysis of the ass…
Hierarchically Compositional Tasks and Deep Convolutional Networks
Arturo Deza, Qianli Liao, Andrzej Banburski +1
The main success stories of deep learning, starting with ImageNet, depend on deep convolutional networks, which on certain tasks perform significantly better than traditional shall…
Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization
Tomaso Poggio, Andrzej Banburski, Qianli Liao
While deep learning is successful in a number of applications, it is not yet well understood theoretically. A satisfactory theoretical characterization of deep learning however, is…
Theory III: Dynamics and Generalization in Deep Networks
Andrzej Banburski, Qianli Liao, Brando Miranda +4
The key to generalization is controlling the complexity of the network. However, there is no obvious control of complexity -- such as an explicit regularization term -- in the trai…
Biologically-plausible learning algorithms can scale to large datasets
Will Xiao, Honglin Chen, Qianli Liao +1
The backpropagation (BP) algorithm is often thought to be biologically implausible in the brain. One of the main reasons is that BP requires symmetric weight matrices in the feedfo…
A Surprising Linear Relationship Predicts Test Performance in Deep Networks
Qianli Liao, Brando Miranda, Andrzej Banburski +2
Given two networks with the same training loss on a dataset, when would they have drastically different test losses and errors? Better understanding of this question of generalizat…
Theory IIIb: Generalization in Deep Networks
Tomaso Poggio, Qianli Liao, Brando Miranda +3
A main puzzle of deep neural networks (DNNs) revolves around the apparent absence of "overfitting", defined in this paper as follows: the expected error does not get worse when inc…
Theory of Deep Learning III: explaining the non-overfitting puzzle
Tomaso Poggio, Kenji Kawaguchi, Qianli Liao +5
A main puzzle of deep networks revolves around the absence of overfitting despite large overparametrization and despite the large capacity demonstrated by zero training error on ra…
Theory of Deep Learning IIb: Optimization Properties of SGD
Chiyuan Zhang, Qianli Liao, Alexander Rakhlin +3
In Theory IIb we characterize with a mix of theory and experiments the optimization of deep convolutional networks by Stochastic Gradient Descent. The main new result in this paper…
Theory II: Landscape of the Empirical Risk in Deep Learning
Qianli Liao, Tomaso Poggio
Previous theoretical work on deep learning and neural network optimization tend to focus on avoiding saddle points and local minima. However, the practical observation is that, at…
Compression of Deep Neural Networks for Image Instance Retrieval
Vijay Chandrasekhar, Jie Lin, Qianli Liao +4
Image instance retrieval is the problem of retrieving images from a database which contain the same object. Convolutional Neural Network (CNN) based descriptors are becoming the do…
Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review
Tomaso Poggio, Hrushikesh Mhaskar, Lorenzo Rosasco +2
The paper characterizes classes of functions for which deep learning can be exponentially better than shallow learning. Deep convolutional networks are a special case of these cond…
Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent Learning
Qianli Liao, Kenji Kawaguchi, Tomaso Poggio
We systematically explored a spectrum of normalization algorithms related to Batch Normalization (BN) and propose a generalized formulation that simultaneously solves two major lim…
View-tolerant face recognition and Hebbian learning imply mirror-symmetric neural tuning to head orientation
Joel Z. Leibo, Qianli Liao, Winrich Freiwald +2
The primate brain contains a hierarchy of visual areas, dubbed the ventral stream, which rapidly computes object representations that are both specific for object identity and rela…
Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex
Qianli Liao, Tomaso Poggio
We discuss relations between Residual Networks (ResNet), Recurrent Neural Networks (RNNs) and the primate visual cortex. We begin with the observation that a special type of shallo…
Learning Functions: When Is Deep Better Than Shallow
Hrushikesh Mhaskar, Qianli Liao, Tomaso Poggio
While the universal approximation property holds both for hierarchical and shallow networks, we prove that deep (hierarchical) networks can approximate the class of compositional f…
How Important is Weight Symmetry in Backpropagation?
Qianli Liao, Joel Z. Leibo, Tomaso Poggio
Gradient backpropagation (BP) requires symmetric feedforward and feedback connections -- the same weights must be used for forward and backward passes. This "weight transport probl…
Unsupervised learning of clutter-resistant visual representations from natural videos
Qianli Liao, Joel Z. Leibo, Tomaso Poggio
Populations of neurons in inferotemporal cortex (IT) maintain an explicit code for object identity that also tolerates transformations of object appearance e.g., position, scale, v…
Can a biologically-plausible hierarchy effectively replace face detection, alignment, and recognition pipelines?
Qianli Liao, Joel Z Leibo, Youssef Mroueh +1
The standard approach to unconstrained face recognition in natural photographs is via a detection, alignment, recognition pipeline. While that approach has achieved impressive resu…