From the 1 of 4 linked papers with an AI index.
4 papers
AMUSE: Anytime Muon with Stable Gradient Evaluation
Jueun Kim, Baekrok Shin, Jihun Yun +3
The paper introduces AMUSE, an optimization method that combines Muon's orthogonal momentum updates with Schedule-Free iterate averaging to achieve fast, stable training without le…
Learning Weakly Communicating Average-Reward CMDPs: Strong Duality and Improved Regret
Kihyun Yu, Beomhan Baek, Dabeen Lee
We study infinite-horizon average-reward constrained Markov decision processes (CMDPs) under the weakly communicating assumption. Our contributions are twofold. First, we establish…
Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime
Beomhan Baek, Minhak Song, Chulhee Yun
Adam [Kingma & Ba, 2015] is the de facto optimizer in deep learning, yet its theoretical understanding remains limited. Prior analyses show that Adam favors solutions aligned with…
Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
Minhak Song, Beomhan Baek, Kwangjun Ahn +1
As both model and dataset sizes continue to scale rapidly, conventional pretraining strategies with fixed compute budgets-such as cosine learning rate schedules-are increasingly in…