2 citations · 5 across the 24 of their papers we have counts for
5 papers · 1 filter
1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillation
Huanxin Sheng, Zhiling Ye, Haonan Wang +3
Sparse on-policy distillation (OPD) allocates teacher supervision to a small subset of tokens in student-generated trajectories. However, useful teacher guidance can yield a noisy…
Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling
Bingguang Hao, Zengzhuang Xu, Maolin Wang +9
The effective training of Large Language Models (LLMs) for function calling faces a critical challenge: balancing exploration of complex reasoning paths with stable policy optimiza…
FunReason: Enhancing Large Language Models' Function Calling via Self-Refinement Multiscale Loss and Automated Data Refinement
Bingguang Hao, ZengZhuang Xu, Maolin Wang +9
The integration of large language models (LLMs) with function calling has emerged as a crucial capability for enhancing their practical utility in real-world applications. However,…
MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts
Zhitian Xie, Yinger Zhang, Chenyi Zhuang +4
The application of mixture-of-experts (MoE) is gaining popularity due to its ability to improve model's performance. In an MoE structure, the gate layer plays a significant role in…
Intelligent Virtual Assistants with LLM-based Process Automation
Yanchu Guan, Dong Wang, Zhixuan Chu +6
While intelligent virtual assistants like Siri, Alexa, and Google Assistant have become ubiquitous in modern life, they still face limitations in their ability to follow multi-step…