4 papers · 1 filter
MemDuet: Learning Dual-Process Memory for Long-Context Reasoning
Jiangcheng Song, Haodong Jing, Yulin Mu +7
Reasoning over infinitely long documents is becoming a reality with recurrent memory agents. However, increasing context length exacerbates the trade-off between reasoning performa…
MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation
Jin Cui, Jiaqi Guo, Jiepeng Zhou +6
While Large Language Models (LLMs) have emerged with remarkable capabilities in complex tasks through Chain-of-Thought reasoning, practical resource constraints have sparked intere…
"The Whole Is Greater Than the Sum of Its Parts": A Compatibility-Aware Multi-Teacher CoT Distillation Framework
Jin Cui, Jiaqi Guo, Ruixuan Yang +6
Chain-of-Thought (CoT) reasoning empowers Large Language Models (LLMs) with remarkable capabilities but typically requires prohibitive parameter scales. CoT distillation has emerge…
SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMs
Haiduo Huang, Jiangcheng Song, Yadong Zhang +1
Knowledge distillation (KD) is a standard route to compress Large Language Models (LLMs) into compact students, yet most pipelines uniformly apply token-wise loss regardless of tea…