2 citations · 5 across the 31 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Most Transformer Modifications Still Do Not Transfer at 1-3B: A 2020-2026 Update to Narang et al. (2021) with Downstream Evaluation and a Noise Floor
Yang Zhao, Jiahao Lu, Bin Huang +2
Narang et al. (2021) evaluated 40+ Transformer modifications at T5-base scale and concluded that most did not transfer. Five years later, the typical working regime has moved to 1-…
cs.LG2025
Frequency Regularization: Unveiling the Spectral Inductive Bias of Deep Neural Networks
Jiahao Lu
Regularization techniques such as L2 regularization (Weight Decay) and Dropout are fundamental to training deep neural networks, yet their underlying physical mechanisms regarding…