597 citations · 2.3k across the 62 of their papers we have counts for
12 papers · 1 filter
Can Vision Transformers Perform Convolution?
Shanda Li, Xiangning Chen, Di He +1
Several recent studies have demonstrated that attention-based networks, such as Vision Transformer (ViT), can outperform Convolutional Neural Networks (CNNs) on several computer vi…
Boosting the Certified Robustness of L-infinity Distance Nets
Bohang Zhang, Du Jiang, Di He +1
Recently, Zhang et al. (2021) developed a new neural network architecture based on -distance functions, which naturally possesses certified robustness by…
Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding
Shengjie Luo, Shanda Li, Tianle Cai +6
The attention module, which is a crucial component in Transformer, cannot scale efficiently to long sequences due to its quadratic complexity. Many works focus on approximating the…
Do Transformers Really Perform Bad for Graph Representation?
Chengxuan Ying, Tianle Cai, Shengjie Luo +5
The Transformer architecture has become a dominant choice in many domains, such as natural language processing and computer vision. Yet, it has not achieved competitive performance…
How could Neural Networks understand Programs?
Dinglan Peng, Shuxin Zheng, Yatao Li +3
Semantic understanding of programs is a fundamental problem for programming language processing (PLP). Recent works that learn representations of code based on pre-training techniq…
Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart
Tianyu Pang, Huishuai Zhang, Di He +5
Correctly classifying adversarial examples is an essential but challenging requirement for safely deploying machine learning models. As reported in RobustBench, even the state-of-t…