1.2k citations · 1.7k across the 28 of their papers we have counts for
9 papers · 1 filter
First Place Solution of KDD Cup 2021 & OGB Large-Scale Challenge Graph Prediction Track
Chengxuan Ying, Mingqi Yang, Shuxin Zheng +7
In this technical report, we present our solution of KDD Cup 2021 OGB Large-Scale Challenge - PCQM4M-LSC Track. We adopt Graphormer and ExpC as our basic models. We train each mode…
Deep Subdomain Adaptation Network for Image Classification
Yongchun Zhu, Fuzhen Zhuang, Jindong Wang +5
For a target task where labeled data is unavailable, domain adaptation can transfer a learner from a different source domain. Previous deep domain adaptation methods mainly learn a…
Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding
Shengjie Luo, Shanda Li, Tianle Cai +6
The attention module, which is a crucial component in Transformer, cannot scale efficiently to long sequences due to its quadratic complexity. Many works focus on approximating the…
Do Transformers Really Perform Bad for Graph Representation?
Chengxuan Ying, Tianle Cai, Shengjie Luo +5
The Transformer architecture has become a dominant choice in many domains, such as natural language processing and computer vision. Yet, it has not achieved competitive performance…
How could Neural Networks understand Programs?
Dinglan Peng, Shuxin Zheng, Yatao Li +3
Semantic understanding of programs is a fundamental problem for programming language processing (PLP). Recent works that learn representations of code based on pre-training techniq…
Transformers with Competitive Ensembles of Independent Mechanisms
Alex Lamb, Di He, Anirudh Goyal +4
An important development in deep learning from the earliest MLPs has been a move towards architectures with structural inductive biases which enable the model to keep distinct sour…