30 citations · 35 across the 7 of their papers we have counts for
5 papers · 1 filter
Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
Xi Chen, Kaituo Feng, Changsheng Li +4
Low-rank training has emerged as a promising approach for reducing memory usage in training Large Language Models (LLMs). Previous methods either rely on decomposing weight matrice…
Shared Growth of Graph Neural Networks via Prompted Free-direction Knowledge Distillation
Kaituo Feng, Yikun Miao, Changsheng Li +2
Knowledge distillation (KD) has shown to be effective to boost the performance of graph neural networks (GNNs), where the typical objective is to distill knowledge from a deeper te…
Towards Open Temporal Graph Neural Networks
Kaituo Feng, Changsheng Li, Xiaolu Zhang +1
Graph neural networks (GNNs) for temporal graphs have recently attracted increasing attentions, where a common assumption is that the class set for nodes is closed. However, in rea…
Robust Knowledge Adaptation for Dynamic Graph Neural Networks
Hanjie Li, Changsheng Li, Kaituo Feng +3
Graph structured data often possess dynamic characters in nature. Recent years have witnessed the increasing attentions paid to dynamic graph neural networks for modelling graph da…
FreeKD: Free-direction Knowledge Distillation for Graph Neural Networks
Kaituo Feng, Changsheng Li, Ye Yuan +1
Knowledge distillation (KD) has demonstrated its effectiveness to boost the performance of graph neural networks (GNNs), where its goal is to distill knowledge from a deeper teache…