10 papers
Polyak-Type Extragradient Methods for Monotone Root-Finding Problems
TaeHo Yoon, Sayantan Choudhury, Ezra Greenberg +1
We study Polyak-type step-size selection for extragradient methods for solving deterministic and stochastic monotone root-finding problems. We show that the known projection-type c…
Multi-source conformal prediction: leveraging heterogeneity via localization
Rohan Hore, Anirban Chatterjee, Sayantan Choudhury
Many modern prediction tasks involve data from multiple heterogeneous sources, while the test distribution may differ substantially from any individual source. Although heterogenei…
Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
Alexander Yukhimchuk, Mladen Kolar, Martin Takáč +1
Gradient clipping is a standard safeguard for training neural networks under noisy, heavy-tailed stochastic gradients; yet, most clipping rules treat all parameters as vectors and…
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2
Most first-order optimizers treat matrix-valued parameters as vectors, ignoring the intrinsic geometry of hidden-layer weights in neural networks. Muon addresses this mismatch by u…
Doubly-Unlinked Regression for Dependent Data
Anik Burman, Sayantan Choudhury, Debangan Dey
Shuffled regression concerns settings in which covariates and responses are observed without their correct pairing. In dependent-data problems, a second form of missing corresponde…
Extragradient Method for -Lipschitz Root-finding Problems
Sayantan Choudhury, Nicolas Loizou
Introduced by Korpelevich in 1976, the extragradient method (EG) has become a cornerstone technique for solving min-max optimization, root-finding problems, and variational inequal…
One-shot Conditional Sampling: MMD meets Nearest Neighbors
Anirban Chatterjee, Sayantan Choudhury, Rohan Hore
How can we generate samples from a conditional distribution that we never fully observe? This question arises across a broad range of applications in both modern machine learning a…
Multiplayer Federated Learning: Reaching Equilibrium with Less Communication
TaeHo Yoon, Sayantan Choudhury, Nicolas Loizou
Traditional Federated Learning (FL) approaches assume collaborative clients with aligned objectives working towards a shared global model. However, in many real-world scenarios, cl…
Methods for Convex -Smooth Optimization: Clipping, Acceleration, and Adaptivity
Eduard Gorbunov, Nazarii Tupitsa, Sayantan Choudhury +4
Due to the non-smoothness of optimization problems in Machine Learning, generalized smoothness assumptions have been gaining a lot of attention in recent years. One of the most pop…
Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad
Sayantan Choudhury, Nazarii Tupitsa, Nicolas Loizou +3
Adaptive methods are extremely popular in machine learning as they make learning rate tuning less expensive. This paper introduces a novel optimization algorithm named KATE, which…