3 citations · 3 across the 6 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024
SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training
Chao Ma, Wenbo Gong, Meyer Scetbon +1
Adaptive optimizers such as Adam (Kingma & Ba, 2015) have been central to the success of large language models. However, they often require to maintain optimizer states throughout…
cs.CR2024★ 3 cited
EdgeLeakage: Membership Information Leakage in Distributed Edge Intelligence Systems
Kongyang Chen, Yi Lin, Hui Luo +4
In contemporary edge computing systems, decentralized edge nodes aggregate unprocessed data and facilitate data analytics to uphold low transmission latency and real-time data proc…
cs.LG2024
Understanding the Generalization Benefits of Late Learning Rate Decay
Yinuo Ren, Chao Ma, Lexing Ying
Why do neural networks trained with large learning rates for a longer time often lead to better generalization? In this paper, we delve into this question by examining the relation…