3 papers
cs.LG2026
Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima
Huanran Chen, Huaqing Zhang, Xiao Li +3
The foundational capabilities of large language models are acquired during pretraining on internet-scale, highly heterogeneous data mixtures. In this work, we investigate an intere…
cs.LG2019
Compiler-Level Matrix Multiplication Optimization for Deep Learning
Huaqing Zhang, Xiaolin Cheng, Hui Zang +1
An important linear algebra routine, GEneral Matrix Multiplication (GEMM), is a fundamental operator in deep learning. Compilers need to translate these routines into low-level cod…
cs.LG2018
Gradient-Coherent Strong Regularization for Deep Neural Networks
Dae Hoon Park, Chiu Man Ho, Yi Chang +1
Regularization plays an important role in generalization of deep neural networks, which are often prone to overfitting with their numerous parameters. L1 and L2 regularizers are co…