From the 1 of 6 linked papers with an AI index.
7 papers
AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding
Hong Liu, Rui Cen, Junhan Shi +10
Speculative decoding accelerates large language model inference without changing the target distribution, but no single drafting structure performs best across real-world workloads…
D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding
Tianyu Liu, Yuhao Shen, Rui Cen +7
The paper introduces D-Cut, an adaptive method that prunes draft tokens across a batch to focus verification on the most promising tokens, improving the speed of speculative decodi…
HIPPO: Accelerating Video Large Language Models Inference via Holistic-aware Parallel Speculative Decoding
Qitan Lv, Tianyu Liu, Wen Wu +4
Speculative decoding (SD) has emerged as a promising approach to accelerate LLM inference without sacrificing output quality. Existing SD methods tailored for video-LLMs primarily…
KALE: Enhancing Knowledge Manipulation in Large Language Models via Knowledge-aware Learning
Qitan Lv, Tianyu Liu, Qiaosheng Zhang +2
Despite the impressive performance of large language models (LLMs) pretrained on vast knowledge corpora, advancing their knowledge manipulation-the ability to effectively recall, r…
Exploiting Edited Large Language Models as General Scientific Optimizers
Qitan Lv, Tianyu Liu, Hong Wang
Large language models (LLMs) have been widely adopted in mathematical optimization in scientific scenarios for their extensive knowledge and advanced reasoning capabilities. Existi…
PEARL: Parallel Speculative Decoding with Adaptive Draft Length
Tianyu Liu, Yun Li, Qitan Lv +4
Speculative decoding (SD), where an extra draft model is employed to provide multiple draft tokens first, and then the original target model verifies these tokens in parallel, has…