6 citations · 9 across the 11 of their papers we have counts for
11 papers · 1 filter
Why Does Adaptive Batching Help LLM Pretraining? A Perspective from Unbounded Variance
Arda Fazla, Antesh Upadhyay, Ege C. Kaya +2
Increasing the batch size during training is a common practice in large language model (LLM) pretraining, yet the theoretical justification behind its success is not well understoo…
Unified High-Probability Analysis of Stochastic Variance-Reduced Estimation
Zhankun Luo, Antesh Upadhyay, M. Berk Sahin +3
Stochastic estimators are fundamental to large-scale optimization, where population quantities must be inferred from noisy oracle observations. Although influential methods such as…
Beyond Bounded Variance: Variance-Reduced Normalized Methods for Nonconvex Optimization under Blum-Gladyshev Noise
Antesh Upadhyay, Arda Fazla, Abolfazl Hashemi
We study nonconvex stochastic optimization under the Blum-Gladyshev (-0) noise model, where the stochastic gradient variance grows quadratically with the distance from…
Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance
Arda Fazla, Ege C. Kaya, Antesh Upadhyay +1
Analysis of Stochastic Gradient Descent (SGD) and its variants typically relies on the assumption of uniformly bounded variance, a condition that frequently fails in practical non-…
RAMPAGE: RAndomized Mid-Point for debiAsed Gradient Extrapolation
Zhankun Luo, M. Berk Sahin, Antesh Upadhyay +2
A celebrated method for Variational Inequalities (VIs) is Extragradient (EG), which can be viewed as a standard discrete-time integration scheme. With this view in mind, in this pa…
First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints
Zhankun Luo, Antesh Upadhyay, Sang Bin Moon +1
This paper addresses the distributed stochastic minimax optimization problem subject to stochastic constraints. We propose a novel first-order Softmax-Weighted Switching Gradient m…
Variance Reduction for Stochastic Switching in Functionally Constrained Federated Optimization
Antesh Upadhyay, Sang Bin Moon, Abolfazl Hashemi +1
Federated learning (FL) is increasingly used in safety-critical and fairness-sensitive applications, where learning objectives must be optimized subject to functional constraints.…
Equitable Federated Learning with Activation Clustering
Antesh Upadhyay, Abolfazl Hashemi
Federated learning is a prominent distributed learning paradigm that incorporates collaboration among diverse clients, promotes data locality, and thus ensures privacy. These clien…
FedNMUT -- Federated Noisy Model Update Tracking Convergence Analysis
Vishnu Pandi Chellapandi, Antesh Upadhyay, Abolfazl Hashemi +1
A novel Decentralized Noisy Model Update Tracking Federated Learning algorithm (FedNMUT) is proposed that is tailored to function efficiently in the presence of noisy communication…
Improved Convergence Analysis and SNR Control Strategies for Federated Learning in the Presence of Noise
Antesh Upadhyay, Abolfazl Hashemi
We propose an improved convergence analysis technique that characterizes the distributed learning paradigm of federated learning (FL) with imperfect/noisy uplink and downlink commu…
On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing
Vishnu Pandi Chellapandi, Antesh Upadhyay, Abolfazl Hashemi +1
Decentralized learning and optimization is a central problem in control that encompasses several existing and emerging applications, such as federated learning. While there exists…