4 papers
Speculative Pipeline Decoding: Higher-Accuracy Drafting with Hidden Latency via Pipeline Parallelism
Yijiong Yu, Huazheng Wang, Shuai Yuan +2
Speculative Decoding (SD) accelerates low-concurrency LLM inference with a draft-then-verify paradigm. Mainstream methods, however, rely on multi-token prediction, which incurs com…
Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio
Yijiong Yu, Shuai Yuan, Jie Zheng +2
Soft context compression reduces the computational workload of processing long contexts in LLMs by encoding long context into a smaller number of latent tokens. However, existing f…
Patience Is The Key to Large Language Model Reasoning
Yijiong Yu
Recent advancements in the field of large language models, particularly through the Chain of Thought (CoT) approach, have demonstrated significant improvements in solving complex p…
Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence
Yijiong Yu
Recent advances in reasoning models have demonstrated significant improvements in accuracy by employing detailed and comprehensive reasoning processes. However, generating these le…