5 citations · 5 across the 4 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Spark Transformer: Reactivating Sparsity in FFN and Attention
Chong You, Kan Wu, Zhipeng Jia +16
The discovery of the lazy neuron phenomenon in trained Transformers, where the vast majority of neurons in their feed-forward networks (FFN) are inactive for each token, has spurre…
cs.LG2025
Scaling Laws for Floating Point Quantization Training
Xingwu Sun, Shuaipeng Li, Ruobing Xie +13
Low-precision training is considered an effective strategy for reducing both training and downstream inference costs. Previous scaling laws for precision mainly focus on integer qu…