activity
20232026
most citedHow Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective

1 citations · 1 across the 8 of their papers we have counts for

collaborators
Showing cs.LGShow all

2 papers · 1 filter

cs.LG2026

Beyond the Matrix Sign: Quadratic Spectral Descent

Qiaozhe Zhang, Jun Sun, Yingzhuang Liu

Muon emerges as a strong competitor of the AdamW for LLM pretraining, because the matrix-wise update it employs can potentially incur smaller second-order penalty than the once dom…

cs.LG2025

Rényi Sharpness: A Novel Sharpness that Strongly Correlates with Generalization

Qiaozhe Zhang, Jun Sun, Ruijie Zhang +1

Sharpness (of the loss minima) is widely believed to be a good indicator of generalization of neural networks. Unfortunately, the correlation between existing sharpness measures an…