16 papers
When Does Generating More Help? Disentangling Fixed-Source Synthesis from Source Expansion in Synthetic Data Scaling
Xu Guo, Jian Tong, Zhihui Lu +1
Synthetic data can be scaled along two routes: Source Expansion (SE), which enlarges the source by adding seed materials or generators, and Fixed-Source Synthesis (FSS), which hold…
MOC: Multi-Order Communication in LLM-based Multi-Agent Systems
Yao Guan, Lin Wang, Zhihu Lu +3
Despite the remarkable progress of Large Language Model (LLM) based Multi-Agent Systems, most research focuses on optimizing coordination topology while largely underexploring the…
Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data
Xu Guo, Runyu Peng, Jian Tong +4
Large language models (LLMs) rely on web-scale corpora for pre-training. The noise inherent in these datasets tends to obscure meaningful patterns and ultimately degrade model perf…
LatentRefusal: Latent-Signal Refusal for Unanswerable Text-to-SQL Queries
Xuancheng Ren, Shijing Hu, Zhihui Lu +2
In LLM-based text-to-SQL systems, unanswerable and underspecified user queries may generate not only incorrect text but also executable programs that yield misleading results or vi…
Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design
Xu Guo, Qiming Ge, Jian Tong +8
Reinforcement Learning with Verifiable Rewards (RLVR) significantly enhances the reasoning capabilities of Large Language Models. When applied to RLVR, Multiple-Choice Questions (M…
Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding
Shijing Hu, Jingyang Li, Zhihui Lu +1
Speculative decoding accelerates large language model (LLM) inference by letting a lightweight draft model propose multiple tokens that the target model verifies in parallel. Yet e…