3 citations · 9 across the 9 of their papers we have counts for
9 papers
Training Deep Learning Models with Norm-Constrained LMOs
Thomas Pethick, Wanyun Xie, Kimon Antonakopoulos +3
In this work, we study optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball. We propose a new stochastic family of algorithms that uses the LMO…
SAMPa: Sharpness-aware Minimization Parallelized
Wanyun Xie, Thomas Pethick, Volkan Cevher
Sharpness-aware minimization (SAM) has been shown to improve the generalization of neural networks. However, each SAM update requires \emph{sequentially} computing two gradients, e…
Improving SAM Requires Rethinking its Optimization Formulation
Wanyun Xie, Fabian Latorre, Kimon Antonakopoulos +2
This paper rethinks Sharpness-Aware Minimization (SAM), which is originally formulated as a zero-sum game where the weights of a network and a bounded perturbation try to minimize/…
Stable Nonconvex-Nonconcave Training via Linear Interpolation
Thomas Pethick, Wanyun Xie, Volkan Cevher
This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the…
Federated Learning under Covariate Shifts with Generalization Guarantees
Ali Ramezani-Kebrya, Fanghui Liu, Thomas Pethick +2
This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, w…
Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems
Thomas Pethick, Puya Latafat, Panagiotis Patrinos +2
This paper introduces a new extragradient-type algorithm for a class of nonconvex-nonconcave minimax problems. It is well-known that finding a local solution for general minimax pr…
Solving stochastic weak Minty variational inequalities without increasing batch size
Thomas Pethick, Olivier Fercoq, Puya Latafat +2
This paper introduces a family of stochastic extragradient-type algorithms for a class of nonconvex-nonconcave problems characterized by the weak Minty variational inequality (MVI)…
Revisiting adversarial training for the worst-performing class
Thomas Pethick, Grigorios G. Chrysos, Volkan Cevher
Despite progress in adversarial training (AT), there is a substantial gap between the top-performing and worst-performing classes in many datasets. For example, on CIFAR10, the acc…
Subquadratic Overparameterization for Shallow Neural Networks
Chaehwan Song, Ali Ramezani-Kebrya, Thomas Pethick +2
Overparameterization refers to the important phenomenon where the width of a neural network is chosen such that learning algorithms can provably attain zero loss in nonconvex train…