910 citations · 2.1k across the 53 of their papers we have counts for
6 papers · 1 filter
Locally adaptive activation functions with slope recovery term for deep and physics-informed neural networks
Ameya D. Jagtap, Kenji Kawaguchi, George Em Karniadakis
We propose two approaches of locally adaptive activation functions namely, layer-wise and neuron-wise locally adaptive activation functions, which improve the performance of deep a…
Gradient Descent Finds Global Minima for Generalizable Deep Neural Networks of Practical Sizes
Kenji Kawaguchi, Jiaoyang Huang
In this paper, we theoretically prove that gradient descent can find a global minimum of non-convex optimization of all layers for nonlinear deep neural networks of sizes commonly…
Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimization
Kenji Kawaguchi, Haihao Lu
We propose a new stochastic optimization framework for empirical risk minimization problems such as those that arise in machine learning. The traditional approaches, such as (mini-…
Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy
Alex Lamb, Vikas Verma, Kenji Kawaguchi +4
Adversarial robustness has become a central goal in deep learning, both in the theory and the practice. However, successful methods to improve the adversarial robustness (such as a…
Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning
Kenji Kawaguchi, Jiaoyang Huang, Leslie Pack Kaelbling
For nonconvex optimization in machine learning, this article proves that every local minimum achieves the globally optimal value of the perturbable gradient basis model at any diff…
Interpolation Consistency Training for Semi-Supervised Learning
Vikas Verma, Kenji Kawaguchi, Alex Lamb +4
We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT…