7 papers
SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD
Dongfang Li, Xiaodong Luo, Ruoyu Sun +62
Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pre…
Why Post-Norm Transformers Collapse: Attention Amplification and Gradient Repair Failure
Xingjian Wang, Qingyu Han, Xiaodong Luo +1
Deep decoder-only Transformers often replace the original Post-Norm architecture with Pre-Norm variants because Post-Norm training is highly sensitive to warmup and learning rate u…
QuadEnhancer: Leveraging Quadratic Transformations to Enhance Deep Neural Networks
Qian Chen, Linxin Yang, Akang Wang +2
The combination of linear transformations and non-linear activation functions forms the foundation of most modern deep neural networks, enabling them to approximate highly complex…
Solving Quadratic Programs via Deep Unrolled Douglas-Rachford Splitting
Jinxin Xiong, Xi Gao, Linxin Yang +3
Convex quadratic programs (QPs) are fundamental to numerous applications, including finance, engineering, and energy systems. Among the various methods for solving them, the Dougla…
When GNNs meet symmetry in ILPs: an orbit-based feature augmentation approach
Qian Chen, Lei Li, Qian Li +6
A common characteristic in integer linear programs (ILPs) is symmetry, allowing variables to be permuted without altering the underlying problem structure. Recently, GNNs have emer…
An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling
Linxin Yang, Bingheng Li, Tian Ding +6
Quadratic programs (QPs) arise in various domains such as machine learning, finance, and control. Recently, learning-enhanced primal-dual hybrid gradient (PDHG) methods have shown…
SymILO: A Symmetry-Aware Learning Framework for Integer Linear Optimization
Qian Chen, Tianjian Zhang, Linxin Yang +5
Integer linear programs (ILPs) are commonly employed to model diverse practical problems such as scheduling and planning. Recently, machine learning techniques have been utilized t…