2 citations · 4 across the 11 of their papers we have counts for
10 papers · 1 filter
Online Experiential Learning for Language Models
Tianzhu Ye, Li Dong, Qingxiu Dong +3
The prevailing paradigm for improving large language models relies on offline training with human annotations or simulated environments, leaving the rich experience accumulated dur…
Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
Yao Tang, Li Dong, Yaru Hao +3
Large language models often solve complex reasoning tasks more effectively with Chain-of-Thought (CoT), but at the cost of long, low-bandwidth token sequences. Humans, by contrast,…
Reinforcement Pre-Training
Qingxiu Dong, Li Dong, Yao Tang +4
In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling paradigm for large language models and reinforcement learning (RL). Specifically, we reframe next-token…
Think Only When You Need with Large Hybrid-Reasoning Models
Lingjie Jiang, Xun Wu, Shaohan Huang +7
Recent Large Reasoning Models (LRMs) have shown substantially improved reasoning capabilities over traditional Large Language Models (LLMs) by incorporating extended thinking proce…
Reward Reasoning Model
Jiaxin Guo, Zewen Chi, Li Dong +4
Reward models play a critical role in guiding large language models toward outputs that align with human expectations. However, an open challenge remains in effectively utilizing t…
Scaling Laws of Synthetic Data for Language Models
Zeyu Qin, Qingxiu Dong, Xingxing Zhang +10
Large language models (LLMs) achieve strong performance across diverse tasks, largely driven by high-quality web data used in pre-training. However, recent studies indicate this da…
MPO: Boosting LLM Agents with Meta Plan Optimization
Weimin Xiong, Yifan Song, Qingxiu Dong +4
Recent advancements in large language models (LLMs) have enabled LLM-based agents to successfully tackle interactive planning tasks. However, despite their successes, existing appr…
Self-Boosting Large Language Models with Synthetic Preference Data
Qingxiu Dong, Li Dong, Xingxing Zhang +2
Through alignment with human preferences, Large Language Models (LLMs) have advanced significantly in generating honest, harmless, and helpful responses. However, collecting high-q…
Data Selection via Optimal Control for Language Models
Yuxian Gu, Li Dong, Hongning Wang +4
This work investigates the selection of high-quality pre-training data from massive corpora to enhance LMs' capabilities for downstream usage. We formulate data selection as a gene…
Towards Optimal Learning of Language Models
Yuxian Gu, Li Dong, Yaru Hao +3
This work studies the general principles of improving the learning of language models (LMs), which aims at reducing the necessary training steps for achieving superior performance.…