49 citations · 52 across the 2 of their papers we have counts for
8 papers
Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit
Jascha Sohl-Dickstein, Kenji Kawaguchi
Recent work has noted that all bad local minima can be removed from neural network loss landscapes, by adding a single unit with a particular parameterization. We show that the cor…
Elimination of All Bad Local Minima in Deep Learning
Kenji Kawaguchi, Leslie Pack Kaelbling
In this paper, we theoretically prove that adding one special neuron per output unit eliminates all suboptimal local minima of any deep neural network, for multi-class classificati…
Effect of Depth and Width on Local Minima in Deep Learning
Kenji Kawaguchi, Jiaoyang Huang, Leslie Pack Kaelbling
In this paper, we analyze the effects of depth and width on the quality of local minima, without strong over-parameterization and simplification assumptions in the literature. With…
Depth with Nonlinearity Creates No Bad Local Minima in ResNets
Kenji Kawaguchi, Yoshua Bengio
In this paper, we prove that depth with nonlinearity creates no bad local minima in a type of arbitrarily deep ResNets with arbitrary nonlinear activation functions, in the sense t…
Generalization in Machine Learning via Analytical Learning Theory
Kenji Kawaguchi, Yoshua Bengio, Vikas Verma +1
This paper introduces a novel measure-theoretic theory for machine learning that does not require statistical assumptions. Based on this theory, a new regularization method in deep…
Theory of Deep Learning III: explaining the non-overfitting puzzle
Tomaso Poggio, Kenji Kawaguchi, Qianli Liao +5
A main puzzle of deep networks revolves around the absence of overfitting despite large overparametrization and despite the large capacity demonstrated by zero training error on ra…