6 papers · 1 filter
EasyOPD: An Easy-to-use On-Policy Distillation Framework for Large Language Models
Jie Sun, Mao Zheng, Mingyang Song +7
Conventional language-model distillation often relies on fixed teacher-generated data, which may not cover the states encountered by an evolving student policy. On-policy distillat…
SimCT: Recovering Lost Supervision for Cross-Tokenizer On-Policy Distillation
Jie Sun, Mao Zheng, Mingyang Song +6
On-policy distillation (OPD) is a standard tool for transferring teacher behavior to a smaller student, but it implicitly assumes that teacher and student predictions are comparabl…
SepSeq: A Training-Free Framework for Long Numerical Sequence Processing in LLMs
Jie Sun, Yu Liu, Lu Han +9
While transformer-based Large Language Models (LLMs) theoretically support massive context windows, they suffer from severe performance degradation when processing long numerical s…
SCALE: Selective Resource Allocation for Overcoming Performance Bottlenecks in Mathematical Test-time Scaling
Yang Xiao, Chunpu Xu, Ruifeng Yuan +3
Test-time compute scaling has emerged as a powerful paradigm for enhancing mathematical reasoning in large language models (LLMs) by allocating additional computational resources d…
Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal Evolution of Human States
Yang Xiao, Jiashuo Wang, Qiancheng Xu +5
As Large Language Models (LLMs) increasingly participate in human-AI interactions, evaluating their Theory of Mind (ToM) capabilities - particularly their ability to track dynamic…
LIMOPro: Reasoning Refinement for Efficient and Effective Test-time Scaling
Yang Xiao, Jiashuo Wang, Ruifeng Yuan +4
Large language models (LLMs) have demonstrated remarkable reasoning capabilities through test-time scaling approaches, particularly when fine-tuned with chain-of-thought (CoT) data…