9 papers
AdaGC: Enhancing LLM Pretraining Stability via Adaptive Gradient Clipping
Guoxia Wang, Shuai Li, Congliang Chen +5
Loss spikes remain a persistent obstacle in large-scale language model pretraining. While previous research has attempted to identify the root cause of loss spikes by investigating…
Off-Policy Value-Based Reinforcement Learning for Large Language Models
Peng-Yuan Wang, Ziniu Li, Tian Xu +8
Improving data utilization efficiency is critical for scaling reinforcement learning (RL) for long-horizon tasks where generating trajectories is expensive. However, the dominant R…
Adam Converges Without Any Modification On Update Rules
Yushun Zhang, Bingran Li, Congliang Chen +2
Adam is the default algorithm for training neural networks, including large language models (LLMs). However, \citet{reddi2019convergence} provided an example that Adam diverges, ra…
ORGEval: Graph-Theoretic Evaluation of LLMs in Optimization Modeling
Zhuohan Wang, Ziwei Zhu, Ziniu Li +8
Formulating optimization problems for industrial applications demands significant manual effort and domain expertise. While Large Language Models (LLMs) show promise in automating…
Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
Ziniu Li, Congliang Chen, Tianyun Yang +5
Large Language Models (LLMs) can self-improve through reinforcement learning, where they generate trajectories to explore and discover better solutions. However, this exploration p…
Rethinking Data Mixture for Large Language Models: A Comprehensive Survey and New Perspectives
Yajiao Liu, Congliang Chen, Junchi Yang +1
Training large language models with data collected from various domains can improve their performance on downstream tasks. However, given a fixed training budget, the sampling prop…