57 citations · 68 across the 10 of their papers we have counts for
Showing 2022Show all
3 papers · 1 filter
cs.CL2022★ 2 cited
Finding Skill Neurons in Pre-trained Transformer-based Language Models
Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang +3
Transformer-based pre-trained language models have demonstrated superior performance on various natural language processing tasks. However, it remains unclear how the skills requir…
cs.LG2022★ 3 cited
How Does Sharpness-Aware Minimization Minimize Sharpness?
Kaiyue Wen, Tengyu Ma, Zhiyuan Li
Sharpness-Aware Minimization (SAM) is a highly effective regularization technique for improving the generalization of deep neural networks for various settings. However, the underl…
cs.LG2022★ 3 cited
Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models
Kaiyue Wen, Jiaye Teng, Jingzhao Zhang
Studies on benign overfitting provide insights for the success of overparameterized deep learning models. In this work, we examine whether overfitting is truly benign in real-world…