7 papers
POP: Prefill-Only Pruning for Efficient Large Model Inference
Junhui He, Zhihui Fu, Jun Wang +1
Large Language Models (LLMs) and Vision-Language Models (VLMs) have demonstrated remarkable capabilities. However, their deployment is hindered by significant computational costs.…
Knowledge-Driven Multi-Turn Jailbreaking on Large Language Models
Songze Li, Ruishi He, Xiaojun Jia +2
Large Language Models (LLMs) face a significant threat from multi-turn jailbreak attacks, where adversaries progressively steer conversations to elicit harmful outputs. However, th…
When, What, and How: Rethinking Retrieval-Enhanced Speculative Decoding
Min Fang, Zhihui Fu, Qibin Zhao +1
Speculative decoding (SD) has emerged as an effective technique to accelerate large language model (LLM) inference without compromising output quality. However, the achievable spee…
DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding
Guanghao Li, Zhihui Fu, Min Fang +4
As large language models (LLMs) scale up, accuracy improves, but the autoregressive (AR) nature of decoding increases latency since each token requires a serial forward pass. Specu…
GreedyPrune: Retenting Critical Visual Token Set for Large Vision Language Models
Ruiguang Pei, Weiqing Sun, Zhihui Fu +1
Although Large Vision Language Models (LVLMs) have demonstrated remarkable performance in image understanding tasks, their computational efficiency remains a significant challenge,…
Accelerating Prefilling for Long-Context LLMs via Sparse Pattern Sharing
Dan Peng, Zhihui Fu, Zewen Ye +2
Sparse attention methods exploit the inherent sparsity in attention to speed up the prefilling phase of long-context inference, mitigating the quadratic complexity of full attentio…