From the 1 of 5 linked papers with an AI index.
5 papers
ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation
Qingyu Zhang, Qianhao Yuan, Hongyu Lin +7
The paper proposes ShortOPD, a short-to-long on-policy distillation method that recovers the generation quality of structured-pruned large language models by focusing training on e…
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
Shijie Cao, Qingyu Zhang, Boxi Yu +6
Omni modal large language models (OmniLLMs) have attracted wide attention for their ability to jointly process audio and video, but they generate large token sequences under audio-…
Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation
Luxi Lin, Zhihang Lin, Zhanpeng Zeng +5
Speculative decoding accelerates LLM inference but suffers from performance degradation when target models are fine-tuned for specific domains. A naive solution is to retrain draft…
AI-Salesman: Towards Reliable Large Language Model Driven Telemarketing
Qingyu Zhang, Chunlei Xin, Xuanang Chen +7
Goal-driven persuasive dialogue, exemplified by applications like telemarketing, requires sophisticated multi-turn planning and strict factual faithfulness, which remains a signifi…
ShortV: Efficient Multimodal Large Language Models by Freezing Visual Tokens in Ineffective Layers
Qianhao Yuan, Qingyu Zhang, Yanjiang Liu +6
Multimodal Large Language Models (MLLMs) suffer from high computational costs due to their massive size and the large number of visual tokens. In this paper, we investigate layer-w…