3 papers
cs.LG2025
From Task-Specific Models to Unified Systems: A Review of Model Merging Approaches
Wei Ruan, Tianze Yang, Yifan Zhou +2
Model merging has achieved significant success, with numerous innovative methods proposed to enhance capabilities by combining multiple models. However, challenges persist due to t…
cs.CL2024
Towards Next-Generation Medical Agent: How o1 is Reshaping Decision-Making in Medical Scenarios
Shaochen Xu, Yifan Zhou, Zhengliang Liu +19
Artificial Intelligence (AI) has become essential in modern healthcare, with large language models (LLMs) offering promising advances in clinical decision-making. Traditional model…
cs.LG2024
HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization
Huaqin Zhao, Jiaxi Li, Yi Pan +7
Fine-tuning large language models (LLMs) poses significant memory challenges, as the back-propagation process demands extensive resources, especially with growing model sizes. Rece…