2 citations · 3 across the 5 of their papers we have counts for
6 papers
Diagnosing On-Policy Self-Distillation for Reasoning Language Models
Yang Li, Gongle Xue, Yuheng Yuan +4
On-policy self-distillation (OPSD) has attracted growing interest as a promising approach to improve the reasoning ability of language models. Without external rewards nor a separa…
CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPO
Yang Li, Gongle Xue, Yijia Guo +3
Reinforcement learning with verifiable rewards (RLVR), especially Group Relative Policy Optimization (GRPO), has been widely used to improve reasoning in large language models. How…
Aligning Instruction Tuning with Pre-training
Yiming Liang, Tianyu Zheng, Xinrun Du +12
Instruction tuning enhances large language models (LLMs) to follow human instructions across diverse tasks, relying on high-quality datasets to guide behavior. However, these datas…
I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm
Yiming Liang, Ge Zhang, Xingwei Qu +9
Large Language Models (LLMs) have achieved significant advancements, however, the common learning paradigm treats LLMs as passive information repositories, neglecting their potenti…
TEGEE: Task dEfinition Guided Expert Ensembling for Generalizable and Few-shot Learning
Xingwei Qu, Yiming Liang, Yucheng Wang +10
Large Language Models (LLMs) exhibit the ability to perform in-context learning (ICL), where they acquire new tasks directly from examples provided in demonstrations. This process…
m2mKD: Module-to-Module Knowledge Distillation for Modular Transformers
Ka Man Lo, Yiming Liang, Wenyu Du +5
Modular neural architectures are gaining attention for their powerful generalization and efficient adaptation to new domains. However, training these models poses challenges due to…