2 citations · 2 across the 10 of their papers we have counts for
6 papers · 1 filter
SE-Merging: A Self-Enhanced Approach for Dynamic Model Merging
Zijun Chen, Zhanpeng Zhou, Bo Zhang +3
Model merging has gained increasing attention due to its intriguing property: interpolating the parameters of different task-specific fine-tuned models leads to multi-task abilitie…
ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMs
Feng He, Zijun Chen, Xinnian Liang +4
Recent advances in Large Reasoning Models (LRMs) trained with Long Chain-of-Thought (Long CoT) reasoning have demonstrated remarkable cross-domain generalization capabilities. Howe…
On the Role of Label Noise in the Feature Learning Process
Andi Han, Wei Huang, Zhanpeng Zhou +5
Deep learning with noisy labels presents significant challenges. In this work, we theoretically characterize the role of label noise from a feature learning perspective. Specifical…
New Evidence of the Two-Phase Learning Dynamics of Neural Networks
Zhanpeng Zhou, Yongyi Yang, Mahito Sugiyama +1
Understanding how deep neural networks learn remains a fundamental challenge in modern machine learning. A growing body of evidence suggests that training dynamics undergo a distin…
On the Cone Effect in the Learning Dynamics
Zhanpeng Zhou, Yongyi Yang, Jie Ren +2
Understanding the learning dynamics of neural networks is a central topic in the deep learning community. In this paper, we take an empirical perspective to study the learning dyna…
The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training
Jinbo Wang, Mingze Wang, Zhanpeng Zhou +3
Transformers consist of diverse building blocks, such as embedding layers, normalization layers, self-attention mechanisms, and point-wise feedforward networks. Thus, understanding…