27 papers
Simple-OPD: Demystifying Warm-up for On-policy Distillation
Tao Liu, Taiqiang Wu, Mao Zheng +5
On-policy distillation (OPD) trains a student on its own rollouts with token-level supervision from teacher models, but its effectiveness can depend strongly on the warm-up stage b…
NGM-RAG: Neural Graph Matching based Retrieval-Augmented Generation
Guo Chen, Ziwen Li, Maolin Zheng +3
The paper proposes NGM-RAG, a framework that combines graph neural networks with text matching to improve retrieval-augmented generation for tasks requiring multi-hop reasoning and…
EasyOPD: An Easy-to-use On-Policy Distillation Framework for Large Language Models
Jie Sun, Mao Zheng, Mingyang Song +7
The paper introduces EasyOPD, a modular framework that simplifies on-policy distillation for large language models by separating configuration, supervision logic, and distributed e…
A Survey of On-Policy Distillation for Large Language Models
Mingyang Song, Mao Zheng
As Large Language Models continue to grow in both capability and cost, transferring frontier capabilities into smaller, deployable students has become an important engineering prob…
On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents
Gengsheng Li, Mao Zheng, Mingyang Song +8
Multi-turn agents that plan, invoke tools, and interact with environments offer a promising paradigm for solving complex tasks, yet their capabilities typically rely on very large…
Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents
Tianxiang Fei, Mingyang Song, Mao Zheng +1
Long-term memory for an LLM agent is more than retrieving the right passage at the right time. Current memory systems collapse belief revision, causal coupling, and cross-domain ab…