9 papers
Look Before You Select: Rethinking Vocabulary Sparsification in On-Policy Distillation
Yongliang Miao, Shuang Liu, Yanguang Liu +2
On-policy distillation (OPD) uses teacher correction on student-generated responses. Full-vocabulary correction can provide important corrections even for tokens that the student a…
BiasReducer: Adaptive Bias Mitigation for Reward Models
Shuang Liu, Yongliang Miao, Yanguang Liu +2
Reward models score responses from large language models (LLMs) and guide LLM training toward human preferences. However, reward models can favor superficial attributes such as len…
DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity
Fengyuan Liu, Yongliang Miao, Zirui He +3
Reward models trained from pairwise preferences often exploit superficial shortcut cues rather than learning true response quality. We propose DynaCF, a dynamic reweighting framewo…
RASFT: Rollout-Adaptive Supervised Fine-Tuning for Reasoning
Yongliang Miao, Fengyuan Liu, Wei Shi +4
Supervised fine-tuning (SFT) is a prevailing method for adapting large language models to reasoning tasks by imitating offline expert demonstrations, often treating a single expert…
SkillLens: Adaptive Multi-Granularity Skill Reuse for Cost-Efficient LLM Agents
Yongliang Miao, Ziyang Yu, Liang Zhao +2
Skill libraries have become a practical way for LLM agents to reuse procedural experience across tasks. However, existing systems typically treat skills as flat, single-resolution…
NeuronScope: A Multi-Agent Framework for Explaining Polysemantic Neurons in Language Models
Weiqi Liu, Yongliang Miao, Haiyan Zhao +2
Neuron-level interpretation in large language models (LLMs) is fundamentally challenged by widespread polysemanticity, where individual neurons respond to multiple distinct semanti…
AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling
Yongliang Miao, Yangyang Liang, Mengnan Du
Reward modeling is essential for aligning large language models with human preferences, yet predominant architectures rely on a static pooling strategy to condense sequences into s…
ImagebindDC: Compressing Multi-modal Data with Imagebind-based Condensation
Yue Min, Shaobo Wang, Jiaze Li +5
Data condensation techniques aim to synthesize a compact dataset from a larger one to enable efficient model training, yet while successful in unimodal settings, they often fail in…
CircuitSeer: Mining High-Quality Data by Probing Mathematical Reasoning Circuits in LLMs
Shaobo Wang, Yongliang Miao, Yuancheng Liu +3
Large language models (LLMs) have demonstrated impressive reasoning capabilities, but scaling their performance often relies on massive reasoning datasets that are computationally…