5 citations · 5 across the 10 of their papers we have counts for
Showing 2026 · cs.LGShow all
3 papers · 2 filters
cs.LG2026
Muon Learns More Robust and Transferable Features than Adam
Tianyu Ruan, Fengzhuo Zhang, Shuche Wang +1
Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs) and vision classifiers. Despite its efficiency advantage over Adam and SGD, t…
cs.LG2026
Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic
Hu Tan, Kuo Gai, Shihua Zhang
While neural collapse (NC) predicts that a -class-balanced classifier should organize terminal representations as a -dimensional simplex equiangular tight frame (ETF), mo…
cs.LG2026
Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction
Hu Tan, Kuo Gai, Shihua Zhang
Grokking suggests that fitting the training data and learning a simple underlying rule may occur on different time scales. We formalize this phenomenon by separating the fast decay…