2 citations · 7 across the 33 of their papers we have counts for
34 papers
Agensh: Scaling Organizational Intelligence to 1,024 Agents
Zhihao Zhan, Ting Song, Li Dong +4
A multi-agent system can reduce latency on complex tasks by executing work concurrently. Several pioneering harness frameworks support multi-agent systems. However, the scalability…
Multi-Turn On-Policy Distillation with Prefix Replay
Baohao Liao, Hanze Dong, Christof Monz +3
We study on-policy distillation (OPD) for agentic tasks, where an LLM agent interacts with an environment over multiple turns and a student imitates a teacher over these multi-turn…
VibeVoice-ASR-BitNet Technical Report
Songchen Xu, Ting Song, Shaohan Huang +10
We present VibeVoice-ASR-BitNet, a compressed variant of VibeVoice-ASR optimized for real-time inference on edge CPUs. We apply heterogeneous quantization tailored to the computati…
LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks
Tianzhu Ye, Li Dong, Guanheng Chen +4
Reinforcement learning (RL) on open-ended tasks compresses an LLM's rubric-based evaluation into a scalar reward, discarding rich textual feedback and conflating responses with dis…
You Only Index Once: Cross-Layer Sparse Attention with Shared Routing
Yutao Sun, Yanqi Zhang, Li Dong +2
Long-context inference in modern LLMs is increasingly constrained by decoding efficiency, especially in reasoning-heavy settings where models generate long intermediate chains of t…
Universal YOCO for Efficient Depth Scaling
Yutao Sun, Li Dong, Tianzhu Ye +3
The rise of test-time scaling has remarkably boosted the reasoning and agentic proficiency of Large Language Models (LLMs). Yet, standard Transformers struggle to scale inference-t…
RefineRL: Advancing Competitive Programming with Self-Refinement Reinforcement Learning
Shaopeng Fu, Xingxing Zhang, Li Dong +2
While large language models (LLMs) have demonstrated strong performance on complex reasoning tasks such as competitive programming (CP), existing methods predominantly focus on sin…
On-Policy Context Distillation for Language Models
Tianzhu Ye, Li Dong, Xun Wu +2
Context distillation enables language models to internalize in-context knowledge into their parameters. In our work, we propose On-Policy Context Distillation (OPCD), a framework t…
Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
Zongqian Li, Shaohan Huang, Zewen Chi +5
Modern code generation models exhibit longer outputs, accelerated capability growth, and changed training dynamics, rendering traditional training methodologies, algorithms, and da…
SlideSparse: Fast and Flexible (2N-2):2N Structured Sparsity
Hanyong Shao, Yingbo Hao, Ting Song +10
NVIDIA's 2:4 Sparse Tensor Cores deliver 2x throughput but demand strict 50% pruning -- a ratio that collapses LLM reasoning accuracy (Qwen3: 54% to 15%). Milder patter…
Sparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsity
Di Zhang, Xun Wu, Shaohan Huang +9
Semi-structured N:M sparsity and low-bit quantization (e.g., 1.58-bit BitNet) are two promising approaches for improving the efficiency of large language models (LLMs), yet they ha…
Online Experiential Learning for Language Models
Tianzhu Ye, Li Dong, Qingxiu Dong +3
The prevailing paradigm for improving large language models relies on offline training with human annotations or simulated environments, leaving the rich experience accumulated dur…
Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
Yao Tang, Li Dong, Yaru Hao +3
Large language models often solve complex reasoning tasks more effectively with Chain-of-Thought (CoT), but at the cost of long, low-bandwidth token sequences. Humans, by contrast,…
Computer Environments Elicit General Agentic Intelligence in LLMs
Daixuan Cheng, Shaohan Huang, Yuxian Gu +6
Agentic intelligence in large language models (LLMs) requires not only model intrinsic capabilities but also interactions with external environments. Equipping LLMs with computers…
Black-Box On-Policy Distillation of Large Language Models
Tianzhu Ye, Li Dong, Zewen Chi +3
Black-box distillation creates student large language models (LLMs) by learning from a proprietary teacher model's text outputs alone, without access to its internal logits or para…
The Era of Agentic Organization: Learning to Organize with Language Models
Zewen Chi, Li Dong, Qingxiu Dong +4
We envision a new era of AI, termed agentic organization, where agents solve complex problems by working collaboratively and concurrently, enabling outcomes beyond individual intel…
Thinking Augmented Pre-training
Liang Wang, Nan Yang, Shaohan Huang +2
This paper introduces a simple and scalable approach to improve the data efficiency of large language model (LLM) training by augmenting existing text data with thinking trajectori…
BitNet Distillation
Xun Wu, Shaohan Huang, Wenhui Wang +4
In this paper, we present BitNet Distillation (BitDistill), a lightweight pipeline that fine-tunes off-the-shelf full-precision LLMs (e.g., Qwen) into 1.58-bit precision (i.e., ter…
Information-Preserving Reformulation of Reasoning Traces for Antidistillation
Jiayu Ding, Lei Cui, Li Dong +2
Recent advances in Large Language Models (LLMs) show that extending the length of reasoning chains significantly improves performance on complex tasks. While revealing these reason…
Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
Di Zhang, Xun Wu, Shaohan Huang +6
Recent advances in reinforcement learning (RL) have substantially improved the training of large-scale language models, leading to significant gains in generation quality and reaso…
DocReward: A Document Reward Model for Structuring and Stylizing
Junpeng Liu, Yuzhong Zhao, Bowen Cao +17
Recent agentic workflows automate professional document generation but focus narrowly on textual quality, overlooking structural and stylistic professionalism, which is equally cri…
Benefits and Pitfalls of Reinforcement Learning for Language Model Planning: A Theoretical Perspective
Siwei Wang, Yifei Shen, Haoran Sun +5
Recent reinforcement learning (RL) methods have substantially enhanced the planning capabilities of Large Language Models (LLMs), yet the theoretical basis for their effectiveness…
VibeVoice Technical Report
Zhiliang Peng, Jianwei Yu, Wenhui Wang +10
This report presents VibeVoice, a novel model designed to synthesize long-form speech with multiple speakers by employing next-token diffusion, which is a unified method for modeli…
On-Policy RL with Optimal Reward Baseline
Yaru Hao, Li Dong, Xun Wu +3
Reinforcement learning algorithms are fundamental to align large language models with human preferences and to enhance their reasoning capabilities. However, current reinforcement…