8 papers
OPD: Overlap-Gated On-Policy Distillation on Reliable Prefix
Shuaishuai Zu, Jihao Zhao, Pengfei Li +5
On-policy distillation (OPD) supplies dense token-level supervision by aligning a student to the teacher's next-token distribution on its own generated rollouts. Existing works mos…
TraceSpec: Understanding and Improving Speculative Decoding for LLM Agents
Jihao Zhao, Shuaishuai Zu, Pengfei Li +4
LLM agents execute tasks through repeated model calls over evolving histories of reasoning, actions, and tool feedback. Aggregate speculative-decoding metrics merge these distinct…
Multi-Teacher On-Policy Distillation for Flow-Based Vision-Language-Action Models
Lichao Ni, Yuhan Liu, Pengfei Li +1
Specialized vision-language-action (VLA) policies acquire strong domain-specific skills, yet consolidating these skills into one shared policy remains challenging. On-policy distil…
TreeSpark: Learning to Draft the Trees You Verify for Speculative Decoding
Jihao Zhao, Qianyu Jiang, Shuaishuai Zu +6
Parallel drafters such as DFlash and DSpark reduce proposal latency, but early mismatches limit how much of each block can be accepted. Retaining alternatives introduces parent sta…
Compaction-Aware Reinforcement Learning For Ascend Triton Kernel Generation
Chuansai Zhou, Yanlin Liu, Bingning Huang +7
Automatically generating high-performance kernels with LLM-based agents promises substantial gains in development efficiency, but two obstacles remain. First, while increasing mult…
AscendKernelGen: A Systematic Study of LLM-Based Kernel Generation for Neural Processing Units
Xinzi Cao, Jianyang Zhai, Pengfei Li +17
To meet the ever-increasing demand for computational efficiency, Neural Processing Units (NPUs) have become critical in modern AI infrastructure. However, unlocking their full pote…
SmartSwap: Swap-Based Memory Optimization for LLM Training under Varying Operator Sequences
Zibo Wang, Yuhang Zhou, Zhibin Wang +13
The increasing size of large language models (LLMs) has led to a surge in memory requirements during training, often exceeding the capacity of high-bandwidth memory (HBM). Swap-bas…
Chameleon: Adaptive Fault Tolerance for Distributed Training via Real-time Policy Selection
Yuhang Zhou, Zhibin Wang, Peng Jiang +12
Training large language models faces frequent interruptions due to various faults, demanding robust fault-tolerance. Existing backup-free methods, such as redundant computation, dy…