4 papers
TraceQuery: learning what to recompute from full-denoising trajectory in diffusion language models
Yukai Jin, Zhankai Ye, Jiayue Yang +6
Diffusion large language models (dLLMs) enable flexible generation through iterative masked denoising, but incur substantial inference cost because token representations are repeat…
MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
Zhankai Ye, Yukai Jin, Bingyang Wei +5
Grounding human motion in language, and language in motion, is a central step toward physical AI systems that can understand, generate, and communicate human behavior. Unified moti…
ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning
Shangqian Gao, Ting Hua, Reza Shirkavand +10
Large Language Models (LLMs) have demonstrated remarkable abilities in tackling a wide range of complex tasks. However, their huge computational and memory costs raise significant…
All-in-One Tuning and Structural Pruning for Domain-Specific LLMs
Lei Lu, Zhepeng Wang, Runxue Bao +7
Existing pruning techniques for large language models (LLMs) targeting domain-specific applications typically follow a two-stage process: pruning the pretrained general-purpose LLM…