4 papers
RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
Mengyang Sun, Maochuan Dou, Tao Feng +5
While Large Language Models (LLMs) are commonly fine-tuned to handle domain-specific tasks before being applied to vertical applications, adapting them to complex scenarios with di…
LungCURE: Benchmarking Multimodal Real-World Clinical Reasoning for Precision Lung Cancer Diagnosis and Treatment
Fangyu Hao, Jiayu Yang, Yifan Zhu +14
Lung cancer clinical decision support demands precise reasoning across complex, multi-stage oncological workflows. Existing multimodal large language models (MLLMs) fail to handle…
A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
Mengyang Sun, Yihao Wang, Tao Feng +3
In order to streamline the fine-tuning of foundation models, Low-Rank Adapters (LoRAs) have been substantially adopted across various fields, including instruction tuning and domai…
PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning
Yu Feng, Yangli-ao Geng, Yifan Zhu +7
Federated learning (FL) has gained widespread attention for its privacy-preserving and collaborative learning capabilities. Due to significant statistical heterogeneity, traditiona…