166 citations · 489 across the 33 of their papers we have counts for
6 papers · 1 filter
TuneUp: A Simple Improved Training Strategy for Graph Neural Networks
Weihua Hu, Kaidi Cao, Kexin Huang +4
Despite recent advances in Graph Neural Networks (GNNs), their training strategies remain largely under-explored. The conventional training strategy learns over all nodes in the or…
Gradient Descent Finds Global Minima for Generalizable Deep Neural Networks of Practical Sizes
Kenji Kawaguchi, Jiaoyang Huang
In this paper, we theoretically prove that gradient descent can find a global minimum of non-convex optimization of all layers for nonlinear deep neural networks of sizes commonly…
Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimization
Kenji Kawaguchi, Haihao Lu
We propose a new stochastic optimization framework for empirical risk minimization problems such as those that arise in machine learning. The traditional approaches, such as (mini-…
Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy
Alex Lamb, Vikas Verma, Kenji Kawaguchi +4
Adversarial robustness has become a central goal in deep learning, both in the theory and the practice. However, successful methods to improve the adversarial robustness (such as a…
Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning
Kenji Kawaguchi, Jiaoyang Huang, Leslie Pack Kaelbling
For nonconvex optimization in machine learning, this article proves that every local minimum achieves the globally optimal value of the perturbable gradient basis model at any diff…
Interpolation Consistency Training for Semi-Supervised Learning
Vikas Verma, Kenji Kawaguchi, Alex Lamb +4
We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT…