597 citations · 1.4k across the 38 of their papers we have counts for
4 papers · 2 filters
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
Tianle Cai, Shengjie Luo, Keyulu Xu +3
Normalization is known to help the optimization of deep neural networks. Curiously, different architectures require specialized normalization methods. In this paper, we study what…
Transferred Discrepancy: Quantifying the Difference Between Representations
Yunzhen Feng, Runtian Zhai, Di He +2
Understanding what information neural networks capture is an essential problem in deep learning, and studying whether different models capture similar features is an initial step t…
On Layer Normalization in the Transformer Architecture
Ruibin Xiong, Yunchang Yang, Di He +7
The Transformer is widely used in natural language processing tasks. To train a Transformer however, one usually needs a carefully designed learning rate warm-up stage, which is sh…
MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
Runtian Zhai, Chen Dan, Di He +5
Adversarial training is one of the most popular ways to learn robust models but is usually attack-dependent and time costly. In this paper, we propose the MACER algorithm, which le…