6 papers
HIPPO: Accelerating Video Large Language Models Inference via Holistic-aware Parallel Speculative Decoding
Qitan Lv, Tianyu Liu, Wen Wu +4
Speculative decoding (SD) has emerged as a promising approach to accelerate LLM inference without sacrificing output quality. Existing SD methods tailored for video-LLMs primarily…
KALE: Enhancing Knowledge Manipulation in Large Language Models via Knowledge-aware Learning
Qitan Lv, Tianyu Liu, Qiaosheng Zhang +2
Despite the impressive performance of large language models (LLMs) pretrained on vast knowledge corpora, advancing their knowledge manipulation-the ability to effectively recall, r…
TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees
Tianyu Liu, Qitan Lv, Yuhao Shen +2
Speculative decoding (SD) has become a standard technique for accelerating LLM inference without sacrificing output quality. Recent advances in speculative decoding have shifted fr…
Exploiting Edited Large Language Models as General Scientific Optimizers
Qitan Lv, Tianyu Liu, Hong Wang
Large language models (LLMs) have been widely adopted in mathematical optimization in scientific scenarios for their extensive knowledge and advanced reasoning capabilities. Existi…
PEARL: Parallel Speculative Decoding with Adaptive Draft Length
Tianyu Liu, Yun Li, Qitan Lv +4
Speculative decoding (SD), where an extra draft model is employed to provide multiple draft tokens first, and then the original target model verifies these tokens in parallel, has…
Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models
Qitan Lv, Jie Wang, Hanzhu Chen +3
Generation of plausible but incorrect factual information, often termed hallucination, has attracted significant research interest. Retrieval-augmented language model (RALM) -- whi…