3 papers
cs.CL2026
Multi-Aspect Knowledge Distillation for Language Model with Low-rank Factorization
Zihe Liu, Yulong Mao, Jinan Xu +2
Knowledge distillation is an effective technique for pre-trained language model compression. However, existing methods only focus on the knowledge distribution among layers, which…
cs.CL2025
A Survey on Large Language Models with Multilingualism: Recent Advances and New Frontiers
Kaiyu Huang, Fengran Mo, Xinyu Zhang +10
The rapid development of Large Language Models (LLMs) demonstrates remarkable multilingual capabilities in natural language processing, attracting global attention in both academia…
cs.CL2024
DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution
Yulong Mao, Kaiyu Huang, Changhao Guan +3
Fine-tuning large-scale pre-trained models is inherently a resource-intensive task. While it can enhance the capabilities of the model, it also incurs substantial computational cos…