6 papers
Distillation Traps and Guards: A Calibration Knob for LLM Distillability
Weixiao Zhan, Yongcheng Jing, Leszek Rutkowski +1
Knowledge distillation (KD) transfers capabilities from large language models (LLMs) to smaller students, yet it can fail unpredictably and also underpins model leakage risks. Our…
Multi-Domain Learning with Global Expert Mapping
Pourya Shamsolmoali, Masoumeh Zareapoor, Huiyu Zhou +3
Human perception generalizes well across different domains, but most vision models struggle beyond their training data. This gap motivates multi-dataset learning, where a single mo…
Try, Check and Retry: A Divide-and-Conquer Framework for Boosting Long-context Tool-Calling Performance of LLMs
Kunfeng Chen, Qihuang Zhong, Juhua Liu +2
Tool-calling empowers Large Language Models (LLMs) to interact with external environments. However, current methods often struggle to handle massive and noisy candidate tools in lo…
EvoSyn: Generalizable Evolutionary Data Synthesis for Verifiable Learning
He Du, Bowen Li, Aijun Yang +3
Reliable verifiable data has become a key driver of capability gains in modern language models, enabling stable reinforcement learning with verifiable rewards and effective distill…
Confidence as a Reward: Transforming LLMs into Reward Models
He Du, Bowen Li, Chengxing Xie +3
Reward models can significantly enhance the reasoning capabilities of large language models (LLMs), but they typically require extensive curated data and costly training. To mitiga…
GeometryZero: Advancing Geometry Solving via Group Contrastive Policy Optimization
Yikun Wang, Yibin Wang, Dianyi Wang +4
Recent progress in large language models (LLMs) has boosted mathematical reasoning, yet geometry remains challenging where auxiliary construction is often essential. Prior methods…