6 papers
SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs
Ruyue Liu, Rong Yin, Xiangzhen Bo +5
Large scale pretrained models have revolutionized Natural Language Processing (NLP) and Computer Vision (CV), showcasing remarkable cross domain generalization abilities. However,…
AS-GCL: Asymmetric Spectral Augmentation on Graph Contrastive Learning
Ruyue Liu, Rong Yin, Yong Liu +4
Graph Contrastive Learning (GCL) has emerged as the foremost approach for self-supervised learning on graph-structured data. GCL reduces reliance on labeled data by learning robust…
Communication-Efficient Personalized Federal Graph Learning via Low-Rank Decomposition
Ruyue Liu, Rong Yin, Xiangzhen Bo +5
Federated graph learning (FGL) has gained significant attention for enabling heterogeneous clients to process their private graph data locally while interacting with a centralized…
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
Xunyu Zhu, Jian Li, Can Ma +1
Large Language Models (LLMs) have demonstrated exceptional proficiency in mathematical reasoning tasks due to their extensive parameter counts and training on vast datasets. Despit…
Distilling Mathematical Reasoning Capabilities into Small Language Models
Xunyu Zhu, Jian Li, Yong Liu +2
This work addresses the challenge of democratizing advanced Large Language Models (LLMs) by compressing their mathematical reasoning capabilities into sub-billion parameter Small L…
A Survey on Model Compression for Large Language Models
Xunyu Zhu, Jian Li, Yong Liu +2
Large Language Models (LLMs) have transformed natural language processing tasks successfully. Yet, their large size and high computational needs pose challenges for practical use,…