1 citations · 1 across the 3 of their papers we have counts for
7 papers · 1 filter
Test-Time Scaling in Diffusion LLMs via Hidden Semi-Autoregressive Experts
Jihoon Lee, Hoyeon Moon, Kevin Zhai +6
Diffusion-based large language models (dLLMs) are trained flexibly to model extreme dependence in the data distribution; however, how to best utilize this information at inference…
Federated Multi-Objective Learning with Controlled Pareto Frontiers
Jiansheng Rao, Jiayi Li, Zhizhi Gong +2
Federated learning (FL) is a widely adopted paradigm for privacy-preserving model training, but FedAvg optimise for the majority while under-serving minority clients. Existing meth…
Large Deviation Upper Bounds and Improved MSE Rates of Nonlinear SGD: Heavy-tailed Noise and Power of Symmetry
Aleksandar Armacki, Shuhua Yu, Dragana Bajovic +2
We study large deviation upper bounds and mean-squared error (MSE) guarantees of a general framework of nonlinear stochastic gradient methods in the online setting, in the presence…
Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees
Aleksandar Armacki, Shuhua Yu, Pranay Sharma +4
We study high-probability convergence in online learning, in the presence of heavy-tailed noise. To combat the heavy tails, a general framework of nonlinear SGD methods is consider…
Distributed Sign Momentum with Local Steps for Training Transformers
Shuhua Yu, Ding Zhou, Cong Xie +4
Pre-training Transformer models is resource-intensive, and recent studies have shown that sign momentum is an efficient technique for training large-scale deep learning models, par…
A Unified Framework for Center-based Clustering of Distributed Data
Aleksandar Armacki, Dragana BajoviÄ, DuÅ¡an JakovetiÄ +1
We develop a family of distributed center-based clustering algorithms that work over networks of users. In the proposed scenario, users contain a local dataset and communicate only…