202 citations · 638 across the 18 of their papers we have counts for
8 papers · 1 filter
Auxiliary Learning as an Asymmetric Bargaining Game
Aviv Shamsian, Aviv Navon, Neta Glazer +3
Auxiliary learning is an effective method for enhancing the generalization capabilities of trained models, particularly when dealing with small datasets. However, this approach may…
Robustness Implies Generalization via Data-Dependent Generalization Bounds
Kenji Kawaguchi, Zhun Deng, Kyle Luh +1
This paper proves that robustness implies generalization via data-dependent generalization bounds. As a result, robustness and generalization are shown to be connected closely in a…
Elimination of All Bad Local Minima in Deep Learning
Kenji Kawaguchi, Leslie Pack Kaelbling
In this paper, we theoretically prove that adding one special neuron per output unit eliminates all suboptimal local minima of any deep neural network, for multi-class classificati…
Effect of Depth and Width on Local Minima in Deep Learning
Kenji Kawaguchi, Jiaoyang Huang, Leslie Pack Kaelbling
In this paper, we analyze the effects of depth and width on the quality of local minima, without strong over-parameterization and simplification assumptions in the literature. With…
Theory of Deep Learning III: explaining the non-overfitting puzzle
Tomaso Poggio, Kenji Kawaguchi, Qianli Liao +5
A main puzzle of deep networks revolves around the absence of overfitting despite large overparametrization and despite the large capacity demonstrated by zero training error on ra…
Deep Semi-Random Features for Nonlinear Function Approximation
Kenji Kawaguchi, Bo Xie, Vikas Verma +1
We propose semi-random features for nonlinear function approximation. The flexibility of semi-random feature lies between the fully adjustable units in deep learning and the random…
Depth Creates No Bad Local Minima
Haihao Lu, Kenji Kawaguchi
In deep learning, \textit{depth}, as well as \textit{nonlinearity}, create non-convex loss surfaces. Then, does depth alone create bad local minima? In this paper, we prove that wi…
Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent Learning
Qianli Liao, Kenji Kawaguchi, Tomaso Poggio
We systematically explored a spectrum of normalization algorithms related to Batch Normalization (BN) and propose a generalized formulation that simultaneously solves two major lim…