61 citations · 133 across the 9 of their papers we have counts for
Showing 2022Show all
3 papers · 1 filter
cs.CL2022★ 4 cited
Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers
Zhewei Yao, Xiaoxia Wu, Conglong Li +4
Large-scale transformer models have become the de-facto architectures for various machine learning applications, e.g., CV and NLP. However, those large models also introduce prohib…
cs.CV2022
Rethinking Query-Key Pairwise Interactions in Vision Transformers
Cheng Li, Yangxin Liu
Vision Transformers have achieved state-of-the-art performance in many visual tasks. Due to the quadratic computational and memory complexities of self-attention, recent works eith…
cs.LG2022★ 3 cited
BiFeat: Supercharge GNN Training via Graph Feature Quantization
Yuxin Ma, Ping Gong, Jun Yi +4
Graph Neural Networks (GNNs) is a promising approach for applications with nonEuclidean data. However, training GNNs on large scale graphs with hundreds of millions nodes is both r…