1 citations · 1 across the 6 of their papers we have counts for
6 papers
Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
Jiaming Yan, Jianchun Liu, Hongli Xu +1
Mixture-of-Experts (MoE) has emerged as a promising architecture for modern large language models (LLMs). However, massive parameters impose heavy GPU memory (i.e., VRAM) demands,…
FedQuad: Adaptive Layer-wise LoRA Deployment and Activation Quantization for Federated Fine-Tuning
Rukuo Li, Jianchun Liu, Hongli Xu +1
Federated fine-tuning (FedFT) provides an effective paradigm for fine-tuning large language models (LLMs) in privacy-sensitive scenarios. However, practical deployment remains chal…
Efficient Federated Fine-Tuning of Large Language Models with Layer Dropout
Shilong Wang, Jianchun Liu, Hongli Xu +2
Fine-tuning plays a crucial role in enabling pre-trained LLMs to evolve from general language comprehension to task-specific expertise. To preserve user data privacy, federated fin…
Caesar: A Low-deviation Compression Approach for Efficient Federated Learning
Jiaming Yan, Jianchun Liu, Hongli Xu +4
Compression is an efficient way to relieve the tremendous communication overhead of federated learning (FL) systems. However, for the existing works, the information loss under com…
Fair Differentiable Neural Network Architecture Search for Long-Tailed Data with Self-Supervised Learning
Jiaming Yan
Recent advancements in artificial intelligence (AI) have positioned deep learning (DL) as a pivotal technology in fields like computer vision, data mining, and natural language pro…
Heroes: Lightweight Federated Learning with Neural Composition and Adaptive Local Update in Heterogeneous Edge Networks
Jiaming Yan, Jianchun Liu, Shilong Wang +3
Federated Learning (FL) enables distributed clients to collaboratively train models without exposing their private data. However, it is difficult to implement efficient FL due to l…