4 citations · 7 across the 16 of their papers we have counts for
17 papers
Neural Networks Provably Learn Spectral Representations for Group Composition
Jianliang He, Leda Wang, Fengzhuo Zhang +2
Understanding how structured internal structure emerges during neural network training is central to the study of deep learning. We investigate this phenomenon through the group co…
Supervised Fine-Tuning vs. In-Context Learning: An Equilibrium Analysis of LLM Personalization under Congestion
Fengzhuo Zhang, Zhuoran Yang, Dirk Bergemann
Large Language Models (LLMs) have revolutionized AI services, but a critical tension emerges: while personalization improves model performance, it consumes scarce computational res…
Muon Learns More Robust and Transferable Features than Adam
Tianyu Ruan, Fengzhuo Zhang, Shuche Wang +1
Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs) and vision classifiers. Despite its efficiency advantage over Adam and SGD, t…
Why Muon Outperforms Adam: A Curvature Perspective
Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2
Muon improves training efficiency over Adam in large language-model training by about two times, but the local geometric source of this advantage remains unclear. Our work takes a…
INFUSER: Influence-Guided Self-Evolution Improves Reasoning
Siyu Chen, Miao Lu, Beining Wu +7
Self-evolution offers a scalable path to stronger reasoning: a pretrained language model improves itself with only minimal external supervision. Yet existing methods either depend…
Demystifying the Slash Pattern in Attention: The Role of RoPE
Yuan Cheng, Fengzhuo Zhang, Yunlong Hou +5
Large Language Models (LLMs) often exhibit slash attention patterns, where attention scores concentrate along the -th sub-diagonal for some offset . These patterns play a key…