30 citations · 35 across the 7 of their papers we have counts for
8 papers
Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
Xi Chen, Kaituo Feng, Changsheng Li +4
Low-rank training has emerged as a promising approach for reducing memory usage in training Large Language Models (LLMs). Previous methods either rely on decomposing weight matrice…
Keypoint-based Progressive Chain-of-Thought Distillation for LLMs
Kaituo Feng, Changsheng Li, Xiaolu Zhang +3
Chain-of-thought distillation is a powerful technique for transferring reasoning abilities from large language models (LLMs) to smaller student models. Previous methods typically r…
On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving
Kaituo Feng, Changsheng Li, Dongchun Ren +2
End-to-end motion planning models equipped with deep neural networks have shown great potential for enabling full autonomous driving. However, the oversized neural networks render…
Learning to Generate Parameters of ConvNets for Unseen Image Data
Shiye Wang, Kaituo Feng, Changsheng Li +2
Typical Convolutional Neural Networks (ConvNets) depend heavily on large amounts of image data and resort to an iterative optimization algorithm (e.g., SGD or Adam) to learn networ…
Shared Growth of Graph Neural Networks via Prompted Free-direction Knowledge Distillation
Kaituo Feng, Yikun Miao, Changsheng Li +2
Knowledge distillation (KD) has shown to be effective to boost the performance of graph neural networks (GNNs), where the typical objective is to distill knowledge from a deeper te…
Towards Open Temporal Graph Neural Networks
Kaituo Feng, Changsheng Li, Xiaolu Zhang +1
Graph neural networks (GNNs) for temporal graphs have recently attracted increasing attentions, where a common assumption is that the class set for nodes is closed. However, in rea…
Robust Knowledge Adaptation for Dynamic Graph Neural Networks
Hanjie Li, Changsheng Li, Kaituo Feng +3
Graph structured data often possess dynamic characters in nature. Recent years have witnessed the increasing attentions paid to dynamic graph neural networks for modelling graph da…
FreeKD: Free-direction Knowledge Distillation for Graph Neural Networks
Kaituo Feng, Changsheng Li, Ye Yuan +1
Knowledge distillation (KD) has demonstrated its effectiveness to boost the performance of graph neural networks (GNNs), where its goal is to distill knowledge from a deeper teache…