22 citations · 65 across the 14 of their papers we have counts for
Showing 2022Show all
3 papers · 1 filter
cs.CV2022★ 7 cited
A Generalization of ViT/MLP-Mixer to Graphs
Xiaoxin He, Bryan Hooi, Thomas Laurent +3
Graph Neural Networks (GNNs) have shown great potential in the field of graph representation learning. Standard GNNs define a local message-passing mechanism which propagates infor…
cs.LG2022★ 1 cited
A Study on Transformer Configuration and Training Objective
Fuzhao Xue, Jianghai Chen, Aixin Sun +6
Transformer-based models have delivered impressive results on many tasks, particularly vision and language tasks. In many model training situations, conventional configurations are…
cs.LG2022★ 5 cited
One Student Knows All Experts Know: From Sparse to Dense
Fuzhao Xue, Xiaoxin He, Xiaozhe Ren +2
Human education system trains one student by multiple experts. Mixture-of-experts (MoE) is a powerful sparse architecture including multiple experts. However, sparse MoE model is e…