1 citations · 2 across the 2 of their papers we have counts for
4 papers · 1 filter
Silent Inconsistency in Data-Parallel Full Fine-Tuning: Diagnosing Worker-Level Optimization Misalignment
Hong Li, Zhen Zhou, Honggang Zhang +4
Data-parallel (DP) training with synchronous all-reduce is a dominant paradigm for full-parameter fine-tuning of large language models (LLMs). While parameter synchronization guara…
Towards Robust Zero-Shot Reinforcement Learning
Kexin Zheng, Lauriane Teyssier, Yinan Zheng +2
The recent development of zero-shot reinforcement learning (RL) has opened a new avenue for learning pre-trained generalist policies that can adapt to arbitrary new tasks in a zero…
Omni-Thinker: Scaling Multi-Task RL in LLMs with Hybrid Reward and Task Scheduling
Derek Li, Jiaming Zhou, Leo Maxime Brunswic +8
The pursuit of general-purpose artificial intelligence depends on large language models (LLMs) that can handle both structured reasoning and open-ended generation. We present Omni-…
Step-wise Adaptive Integration of Supervised Fine-tuning and Reinforcement Learning for Task-Specific LLMs
Jack Chen, Fazhong Liu, Naruto Liu +7
Large language models (LLMs) excel at mathematical reasoning and logical problem-solving. The current popular training paradigms primarily use supervised fine-tuning (SFT) and rein…