30 citations · 50 across the 12 of their papers we have counts for
12 papers
BurstEngine: an Efficient Distributed Framework for Training Transformers on Extremely Long Sequences of over 1M Tokens
Ao Sun, Weilin Zhao, Xu Han +4
Existing methods for training LLMs on long-sequence data, such as Tensor Parallelism and Context Parallelism, exhibit low Model FLOPs Utilization as sequence lengths and number of…
Efficient Multi-task Prompt Tuning for Recommendation
Ting Bai, Le Huang, Yue Yu +4
With the expansion of business scenarios, real recommender systems are facing challenges in dealing with the constantly emerging new tasks in multi-task learning frameworks. In thi…
Non-autoregressive Personalized Bundle Generation
Wenchuan Yang, Cheng Yang, Jichao Li +3
The personalized bundle generation problem, which aims to create a preferred bundle for user from numerous candidate items, receives increasing attention in recommendation. However…
Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training
Ao Sun, Weilin Zhao, Xu Han +5
The emergence of large language models (LLMs) relies heavily on distributed training strategies, among which pipeline parallelism plays a crucial role. As LLMs' training sequence l…
FairSIN: Achieving Fairness in Graph Neural Networks through Sensitive Information Neutralization
Cheng Yang, Jixi Liu, Yunhe Yan +1
Despite the remarkable success of graph neural networks (GNNs) in modeling graph-structured data, like other machine learning models, GNNs are also susceptible to making biased pre…
BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences
Ao Sun, Weilin Zhao, Xu Han +4
Effective attention modules have played a crucial role in the success of Transformer-based large language models (LLMs), but the quadratic time and memory complexities of these att…
GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Tasks
Mengmei Zhang, Mingwei Sun, Peng Wang +6
Large language models (LLMs) like ChatGPT, exhibit powerful zero-shot and instruction-following capabilities, have catalyzed a revolutionary transformation across diverse fields, e…
Endowing Pre-trained Graph Models with Provable Fairness
Zhongjian Zhang, Mengmei Zhang, Yue Yu +3
Pre-trained graph models (PGMs) aim to capture transferable inherent structural properties and apply them to different downstream tasks. Similar to pre-trained language models, PGM…
Graph Invariant Learning with Subgraph Co-mixup for Out-Of-Distribution Generalization
Tianrui Jia, Haoyang Li, Cheng Yang +2
Graph neural networks (GNNs) have been demonstrated to perform well in graph representation learning, but always lacking in generalization capability when tackling out-of-distribut…
Graph Foundation Models: Concepts, Opportunities and Challenges
Jiawei Liu, Cheng Yang, Zhiyuan Lu +8
Foundation models have emerged as critical components in a variety of artificial intelligence applications, and showcase significant success in natural language processing and seve…
Graph Distillation with Eigenbasis Matching
Yang Liu, Deyu Bo, Chuan Shi
The increasing amount of graph data places requirements on the efficient training of graph neural networks (GNNs). The emerging graph distillation (GD) tackles this challenge by di…
Data-centric Graph Learning: A Survey
Yuxin Guo, Deyu Bo, Cheng Yang +5
The history of artificial intelligence (AI) has witnessed the significant impact of high-quality data on various deep learning models, such as ImageNet for AlexNet and ResNet. Rece…