4 citations · 11 across the 12 of their papers we have counts for
5 papers · 1 filter
HyperDFlash: Hyper-Connection-Aligned Block Speculative Decoding with Gated Residual Reduction
Luxi Lin, Shuang Peng, Rui Ma +7
We present HyperDFlash, a block-parallel speculative decoding framework tailored to DeepSeek-V4's Hyper-Connections (HC). Despite the strong performance of DeepSeek-V4's native Mul…
S2O: Early Stopping for Sparse Attention via Online Permutation
Yu Zhang, Songwei Liu, Chenqian Yan +4
Attention scales quadratically with sequence length, fundamentally limiting long-context inference. Existing block-granularity sparsification can reduce latency, but coarse blocks…
GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference
Chao Zeng, Songwei Liu, Shu Yang +3
Model compression has emerged as a mainstream solution to reduce memory usage and computational overhead. This paper presents Group Quantization and Sparse Acceleration (GQSA), a n…
ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
Chao Zeng, Songwei Liu, Yusheng Xie +6
Large Language Models (LLMs) have revolutionized natural language processing tasks. However, their practical application is constrained by substantial memory and computational dema…
FoldGPT: Simple and Effective Large Language Model Compression Scheme
Songwei Liu, Chao Zeng, Lianqiang Li +4
The demand for deploying large language models(LLMs) on mobile devices continues to increase, driven by escalating data security concerns and cloud costs. However, network bandwidt…