6 citations · 6 across the 9 of their papers we have counts for
8 papers · 1 filter
DeepFusion: Accelerating MoE Training via Federated Knowledge Distillation from Heterogeneous Edge Devices
Songyuan Li, Jia Hu, Ahmed M. Abdelmoniem +3
Recent Mixture-of-Experts (MoE)-based large language models (LLMs) such as Qwen-MoE and DeepSeek-MoE are transforming generative AI in natural language processing. However, these m…
Preventing Rank Collapse in Federated Low-Rank Adaptation with Client Heterogeneity
Fei Wu, Jia Hu, Geyong Min +1
Federated low-rank adaptation (FedLoRA) has facilitated communication-efficient and privacy-preserving fine-tuning of foundation models for downstream tasks. In practical federated…
BoHA: Blockwise Hadamard Product Adaptation for Parameter-Efficient Fine-Tuning
Feng Yu, Jia Hu, Geyong Min
Parameter-efficient fine-tuning (PEFT) of large language models trains a small task-specific parameter set while keeping the pretrained model frozen. The dominant Low-Rank Adaptati…
Towards Robust Spiking Neural Networks:Mitigating Heterogeneous Training Vulnerability via Dominant Eigencomponent Projection
Desong Zhang, Jia Hu, Geyong Min
Spiking Neural Networks (SNNs) process information via discrete spikes, enabling them to operate at remarkably low energy levels. However, our experimental observations reveal a st…
Task-agnostic Low-rank Residual Adaptation for Efficient Federated Continual Fine-Tuning
Feng Yu, Jia Hu, Geyong Min
Federated Parameter-Efficient Fine-Tuning (Fed-PEFT) enables lightweight adaptation of large pre-trained models in federated learning settings by updating only a small subset of pa…
Efficient Orthogonal Fine-Tuning with Principal Subspace Adaptation
Fei Wu, Jia Hu, Geyong Min +1
Driven by the rapid growth of model parameters, parameter-efficient fine-tuning (PEFT) has become essential for adapting large models to diverse downstream tasks under constrained…