12 papers
REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation
Haoran Que, Jiajun Shi, Ting Huang +7
As language-model compute continues to scale, high-quality training data is becoming an increasingly important bottleneck. Conventional next-token prediction supervises what follow…
Mitigating Rubric Interference in LLM Judges via On-Policy Self-Distillation
Dingyao Yu, Tong Zhang, Yutao Mou +3
LLM judges increasingly evaluate responses against fine-grained rubric checklists. When a sample requires multiple rubrics, current methods typically assess each in a separate infe…
Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models
Zhaoyi Li, Deyang Kong, Yuan Wei +13
On-policy distillation (OPD) transfers teacher capabilities by supervising trajectories sampled from the student's own policy, yet its generalization behavior remains poorly unders…
TrustRoboReward: Preference-Ordered Isotonic Score Editing for Multi-Paradigm Robot Reward Models
Yidong Wang, Yan Zhan, Ziteng Feng +16
Reward models are a bottleneck for reinforcement learning in embodied AI. Long-horizon robotic manipulation requires scalable vision feedback beyond handcrafted rewards or task-spe…
Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubrics
Mengyuan Sun, Yu Li, Zhuohao Yu +2
Rubric-based evaluation is a promising paradigm for judging large language model (LLM) outputs, yet self-generated rubrics lag human-annotated criteria on hard instances. We argue…
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
Yongrui Heng, Chaoya Jiang, Han Yang +2
Self-evolution of multimodal large language models (MLLMs) remains a critical challenge: pseudo-label-based methods suffer from progressive quality degradation as model predictions…
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions
Zhiyin Yu, Yuchen Mou, Juncheng Yan +17
Reinforcement learning (RL) has emerged as a powerful post-training paradigm for enhancing the reasoning capabilities of large language models (LLMs). However, reinforcement learni…
Retrieval as Generation: A Unified Framework with Self-Triggered Information Planning
Bo Li, Mingda Wang, Gexiang Fang +2
We revisit retrieval-augmented generation (RAG) by embedding retrieval control directly into generation. Instead of treating retrieval as an external intervention, we express retri…
Instruction Data Selection via Answer Divergence
Bo Li, Mingda Wang, Shikun Zhang +1
Instruction tuning relies on large instruction-response corpora whose quality and composition strongly affect downstream performance. We propose Answer Divergence-Guided Selection…
Learning from Contrasts: Synthesizing Reasoning Paths from Diverse Search Trajectories
Peiyang Liu, Zhirui Chen, Xi Wang +4
Monte Carlo Tree Search (MCTS) has been widely used for automated reasoning data exploration, but current supervision extraction methods remain inefficient. Standard approaches ret…
Scaling Latent Reasoning via Looped Language Models
Rui-Jie Zhu, Zixuan Wang, Kai Hua +30
Modern LLMs are trained to "think" primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-training and under-leverages pre-training…
Reverse-Engineered Reasoning for Open-Ended Generation
Haozhe Wang, Haoran Que, Qixin Xu +9
While the ``deep reasoning'' paradigm has spurred significant advances in verifiable domains like mathematics, its application to open-ended, creative generation remains a critical…