13 papers
Federated Co-tuning Framework for Large and Small Language Models
Tao Fan, Yan Kang, Guoqiang Ma +4
By adapting Large Language Models (LLMs) to domain-specific tasks or enriching them with domain-specific knowledge, we can fully harness the capabilities of LLMs. Nonetheless, a ga…
Model-based Large Language Model Customization as Service
Zhaomin Wu, Jizhou Guo, Junyi Hou +3
Prominent Large Language Model (LLM) services from providers like OpenAI and Google excel at general tasks but often underperform on domain-specific applications. Current customiza…
FedGRPO: Privately Optimizing Foundation Models with Group-Relative Rewards from Domain Client
Gongxi Zhu, Hanlin Gu, Lixin Fan +2
One important direction of Federated Foundation Models (FedFMs) is leveraging data from small client models to enhance the performance of a large server-side foundation model. Exis…
PPC-GPT: Federated Task-Specific Compression of Large Language Models via Pruning and Chain-of-Thought Distillation
Tao Fan, Guoqiang Ma, Yuanfeng Song +2
Compressing Large Language Models (LLMs) into task-specific Small Language Models (SLMs) encounters two significant challenges: safeguarding domain-specific knowledge privacy and m…
FedCoT: Federated Chain-of-Thought Distillation for Large Language Models
Tao Fan, Weijing Chen, Yan Kang +5
Large Language Models (LLMs) have emerged as a transformative force in artificial intelligence, demonstrating exceptional proficiency across various tasks. However, their deploymen…
Large-Small Model Collaborative Framework for Federated Continual Learning
Hao Yu, Xin Yang, Boyang Fan +4
Continual learning (CL) for Foundation Models (FMs) is an essential yet underexplored challenge, especially in Federated Continual Learning (FCL), where each client learns from a p…