Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Principled Top- Selection for Language Models with Hybrid Gradients
Xuchen Gong, Junfei Sun, Tian Li
Selecting the best items out of candidates is a critical component of modern large language model systems, such as document selection in Retrieval-Augmented Generation (RAG…
cs.LG2026
Asynchronous Heavy-Tailed Optimization
Junfei Sun, Dixi Yao, Xuchen Gong +3
Heavy-tailed stochastic gradient noise, commonly observed in transformer models, can destabilize the optimization process. Recent works mainly focus on developing and understanding…