14 papers
SparseX: Efficient Segment-Level KV Cache Sharing for Interleaved LLM Serving
Quqing Zhang, Kai Chen, Ning Liao +5
In long-context LLM serving, the prefill stage often dominates time-to-first-token and computational cost. Although Prefix Cache in vLLM/PagedAttention has been widely used to reus…
PrefixFlow: Training-Time KV Caching for Schedule-Level Prefix Reuse in LLM RL Training
Pengbo Li, Feiyuan Zhang, Guangming Sheng +8
In reinforcement learning (RL) post-training, one prompt fans out into trajectories, yet standard trainers replay the shared prefix's forward and backward pass times; in lo…
On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
Mind Lab, :, Vin Bo +64
Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state…
Kernel-Smith: A Unified Recipe for Evolutionary Kernel Optimization
He Du, Qiming Ge, Jiakai Hu +18
We present Kernel-Smith, a framework for high-performance GPU kernel and operator generation that combines a stable evaluation-driven evolutionary agent with an evolution-oriented…
TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration
Zerun Ma, Guoqiang Wang, Xinchen Xie +7
While Large Language Models (LLMs) have empowered AI research agents to perform isolated scientific tasks, automating complex, real-world workflows, such as LLM training, remains a…
How to Fine-Tune a Reasoning Model? A Teacher-Student Cooperation Framework to Synthesize Student-Consistent SFT Data
Zixian Huang, Kaichen Yang, Xu Huang +6
A widely adopted strategy for model enhancement is to use synthetic data generated by a stronger model for supervised fine-tuning (SFT). However, for emerging reasoning models like…
Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic
Yichuan Ma, Linyang Li, Yongkang chen +5
As large language models (LLMs) increasingly tackle complex reasoning tasks, test-time scaling has become critical for enhancing capabilities. However, in agentic scenarios with fr…
TL-GRPO: Turn-Level RL for Reasoning-Guided Iterative Optimization
Peiji Li, Linyang Li, Handa Sun +15
Large language models have demonstrated strong reasoning capabilities in complex tasks through tool integration, which is typically framed as a Markov Decision Process and optimize…
Mixing Expert Knowledge: Bring Human Thoughts Back To the Game of Go
Yichuan Ma, Linyang Li, Yongkang Chen +5
Large language models (LLMs) have demonstrated exceptional performance in reasoning tasks such as mathematics and coding, matching or surpassing human capabilities. However, these…
Pre-Trained Policy Discriminators are General Reward Models
Shihan Dou, Shichun Liu, Yuming Yang +19
We offer a novel perspective on reward modeling by formulating it as a policy discriminator, which quantifies the difference between two policies to generate a reward signal, guidi…
Capability Salience Vector: Fine-grained Alignment of Loss and Capabilities for Downstream Task Scaling Law
Qiming Ge, Shuhao Xing, Songyang Gao +8
Scaling law builds the relationship between training computation and validation loss, enabling researchers to effectively predict the loss trending of models across different level…
Code-Driven Inductive Synthesis: Enhancing Reasoning Abilities of Large Language Models with Sequences
Kedi Chen, Zhikai Lei, Fan Zhang +7
Large language models make remarkable progress in reasoning capabilities. Existing works focus mainly on deductive reasoning tasks (e.g., code and math), while another type of reas…
UnitCoder: Scalable Iterative Code Synthesis with Unit Test Guidance
Yichuan Ma, Yunfan Shao, Peiji Li +5
Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, yet code generation remains a major challenge. Current approaches for obtaining high-qualit…
CritiQ: Mining Data Quality Criteria from Human Preferences
Honglin Guo, Kai Lv, Qipeng Guo +8
Language model heavily depends on high-quality data for optimal performance. Existing approaches rely on manually designed heuristics, the perplexity of existing models, training c…