383 citations · 1.2k across the 28 of their papers we have counts for
9 papers · 1 filter
Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)
Yu Huang, Junyang Lin, Chang Zhou +2
Despite the remarkable success of deep multi-modal learning in practice, it has not been well-explained in theory. Recently, it has been observed that the best uni-modal network ou…
Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
Qingsong Lv, Ming Ding, Qiang Liu +7
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understandi…
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Junyang Lin, An Yang, Jinze Bai +9
Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 a…
Learning to Rehearse in Long Sequence Memorization
Zhu Zhang, Chang Zhou, Jianxin Ma +4
Existing reasoning tasks often have an important assumption that the input contents can be always accessed while reasoning, requiring unlimited storage resources and suffering from…
M6-T: Exploring Sparse Expert Models and Beyond
An Yang, Junyang Lin, Rui Men +12
Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scal…
Understanding Negative Sampling in Graph Representation Learning
Zhen Yang, Ming Ding, Chang Zhou +3
Graph representation learning has been extensively studied in recent years. Despite its potential in generating continuous embeddings for various networks, both the effectiveness a…
Learning Disentangled Representations for Recommendation
Jianxin Ma, Chang Zhou, Peng Cui +2
User behavior data in recommender systems are driven by the complex interactions of many latent factors behind the users' decision making processes. The factors are highly entangle…
Dimensional Reweighting Graph Convolutional Networks
Xu Zou, Qiuye Jia, Jianwei Zhang +3
Graph Convolution Networks (GCNs) are becoming more and more popular for learning node representations on graphs. Though there exist various developments on sampling and aggregatio…
Cognitive Knowledge Graph Reasoning for One-shot Relational Learning
Zhengxiao Du, Chang Zhou, Ming Ding +2
Inferring new facts from existing knowledge graphs (KG) with explainable reasoning processes is a significant problem and has received much attention recently. However, few studies…