4 papers
DECA: Decentralizing Block-Wise Adam for Efficient LLM Full-Parameter Fine-Tuning on Non-IID Data
Yunsheng Yuan, Shaowei Li, Kai Wang +5
Fine-tuning large language models (LLMs) in privacy-sensitive and resource-constrained environments remains challenging. Since training data are often distributed across multiple c…
FGRPO: Federated GRPO with Adaptive Aggregation on Non-IID Data
Pengyu Chen, Shaowei Li, Kai Wang +4
Recent advances in language models have established reinforcement learning as the primary paradigm for eliciting self-correction and long-chain reasoning. While group relative poli…
ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion
Rana Muhammad Shahroz Khan, Dongwen Tang, Pingzhi Li +2
Parameter generation has emerged as a novel paradigm for neural network development, offering an alternative to traditional neural network training by synthesizing high-quality mod…
Make Optimization Once and for All with Fine-grained Guidance
Mingjia Shi, Ruihan Lin, Xuxi Chen +8
Learning to Optimize (L2O) enhances optimization efficiency with integrated neural networks. L2O paradigms achieve great outcomes, e.g., refitting optimizer, generating unseen solu…