198 citations · 379 across the 15 of their papers we have counts for
7 papers · 1 filter
Cut your Losses with Squentropy
Like Hui, Mikhail Belkin, Stephen Wright
Nearly all practical neural models for classification are trained using cross-entropy loss. Yet this ubiquitous choice is supported by little theoretical or empirical evidence. Rec…
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach
Mao Ye, Bo Liu, Stephen Wright +2
Bilevel optimization (BO) is useful for solving a variety of important machine learning problems including but not limited to hyperparameter optimization, meta-learning, continual…
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime
Zhiyan Ding, Shi Chen, Qin Li +1
Finding the optimal configuration of parameters in ResNet is a nonconvex minimization problem, but first-order methods nevertheless find the global optimum in the overparameterized…
Overparameterization of deep ResNet: zero loss and mean-field analysis
Zhiyan Ding, Shi Chen, Qin Li +1
Finding parameters in a deep neural network (NN) that fit training data is a nonconvex optimization problem, but a basic first-order optimization method (gradient descent) finds a…
Stochastic Learning for Sparse Discrete Markov Random Fields with Controlled Gradient Approximation Error
Sinong Geng, Zhaobin Kuang, Jie Liu +2
We study the -regularized maximum likelihood estimator/estimation (MLE) problem for discrete Markov random fields (MRFs), where efficient and scalable learning requires both s…
Convergence and Margin of Adversarial Training on Separable Data
Zachary Charles, Shashank Rajput, Stephen Wright +1
Adversarial training is a technique for training robust machine learning models. To encourage robustness, it iteratively computes adversarial examples for the model, and then re-tr…