2 citations · 3 across the 17 of their papers we have counts for
16 papers · 1 filter
Quantifying Depth Sufficiency in Residual Neural Networks: A First-Order Criterion
Zeyu Liu, Jinhao Zhang, Yunquan Zhang +4
How can we determine whether a trained neural network is already deep enough? We study this under a fixed function-preserving residual-growth protocol specifying insertion location…
Mechanisms of Width Scaling in Normalized Residual Networks: The Effective Alignment Dimension
Jinhao Zhang, Zeyu Liu, Zicheng Yan +4
Existing theories of neural-network width characterize asymptotic limits, but provide limited guidance on whether an expansion direction identified from finite training data remain…
DynaTrain: Fast Online Parallelism Switching for Elastic LLM Training
Yuanqing Wang, Yuchen Zhang, Hao Lin +9
Modern large language model (LLM) training is inherently dynamic: resource fluctuations, RLHF phase shifts, and cluster elasticity continually reshape the optimal parallelism layou…
A Qualitative Test-Risk Mechanism for Scaling Behavior in Normalized Residual Networks
Daning Cheng, Zeyu Liu, Jun Sun +4
The scaling behavior, in which test performance often improves as model size and data increase, is a central empirical phenomenon in modern deep learning, yet its theoretical basis…
MoE-DisCo:Low Economy Cost Training Mixture-of-Experts Models
Xin Ye, Daning Cheng, Boyang Zhang +1
Training large-scale Mixture-of-Experts (MoE) models typically requires high-memory, high-bandwidth GPUs (e.g., A100), and their high cost has become a major barrier to large-model…
CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantization
Jinhao Zhang, Yunquan Zhang, Daning Chen +2
Current mainstream post-training quantization methods for large language models typically apply a uniform quantization strategy across all network layers, overlooking the substanti…