2 citations · 4 across the 9 of their papers we have counts for
9 papers
AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding
Hong Liu, Rui Cen, Junhan Shi +10
Speculative decoding accelerates large language model inference without changing the target distribution, but no single drafting structure performs best across real-world workloads…
D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding
Tianyu Liu, Yuhao Shen, Rui Cen +7
Speculative decoding accelerates large language model (LLM) inference without compromising output quality. Recent parallel drafting methods further improve single-request performan…
DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding
Jiebin Zhang, Zhenghan Yu, Song Liu +9
Block diffusion speculative decoding accelerates LLM inference by predicting all tokens within a block simultaneously for the target model to verify in parallel. Predicting an enti…
Learning to Draft: Adaptive Speculative Decoding with Reinforcement Learning
Jiebin Zhang, Zhenghan Yu, Liang Wang +8
Speculative decoding accelerates large language model (LLM) inference by using a small draft model to generate candidate tokens for a larger target model to verify. The efficacy of…
Hierarchical Memory Organization for Wikipedia Generation
Eugene J. Yu, Dawei Zhu, Yifan Song +6
Generating Wikipedia articles autonomously is a challenging task requiring the integration of accurate, comprehensive, and well-structured information from diverse sources. This pa…
A Comprehensive Survey on Long Context Language Modeling
Jiaheng Liu, Dawei Zhu, Zhiqi Bai +34
Efficient processing of long contexts has been a persistent pursuit in Natural Language Processing. With the growing number of long documents, dialogues, and other textual data, it…
More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
Jiebin Zhang, Dawei Zhu, Yifan Song +6
As large language models (LLMs) process increasing context windows, the memory usage of KV cache has become a critical bottleneck during inference. The mainstream KV compression me…
WIKIGENBENCH: Exploring Full-length Wikipedia Generation under Real-World Scenario
Jiebin Zhang, Eugene J. Yu, Qinyu Chen +8
It presents significant challenges to generate comprehensive and accurate Wikipedia articles for newly emerging events under a real-world scenario. Existing attempts fall short eit…
ConFiguRe: Exploring Discourse-level Chinese Figures of Speech
Dawei Zhu, Qiusi Zhan, Zhejian Zhou +3
Figures of speech, such as metaphor and irony, are ubiquitous in literature works and colloquial conversations. This poses great challenge for natural language understanding since…