1 citations · 2 across the 18 of their papers we have counts for
11 papers · 1 filter
Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match
Jinze Li, Yixing Xu, Guanchen Li +6
Large language models (LLMs) achieve strong performance across diverse tasks but suffer from high inference latency due to their autoregressive generation. Speculative Decoding (SP…
SpecVLM: Fast Speculative Decoding in Vision-Language Models
Haiduo Huang, Fuwei Yang, Zhenhua Liu +4
Speculative decoding is a powerful way to accelerate autoregressive large language models (LLMs), but directly porting it to vision-language models (VLMs) faces unique systems cons…
SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning
Huanxuan Liao, Yixing Xu, Shizhu He +6
Long-context inference in large language models (LLMs) is increasingly constrained by the KV cache bottleneck: memory usage grows linearly with sequence length, while attention com…
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models
Wenjin Ke, Zhe Li, Dong Li +2
Improving the efficiency of inference in Large Language Models (LLMs) is a critical area of research. Post-training Quantization (PTQ) is a popular technique, but it often faces ch…
MonoGS++: Fast and Accurate Monocular RGB Gaussian SLAM
Renwu Li, Wenjing Ke, Dong Li +2
We present MonoGS++, a novel fast and accurate Simultaneous Localization and Mapping (SLAM) method that leverages 3D Gaussian representations and operates solely on RGB inputs. Whi…
PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation
Zihao An, Huajun Bai, Ziqiong Liu +2
The autoregressive nature of large language models (LLMs) fundamentally limits inference speed, as each forward pass generates only a single token and is often bottlenecked by memo…
Partial Convolution Meets Visual Attention
Haiduo Huang, Fuwei Yang, Dong Li +5
Designing an efficient and effective neural network has remained a prominent topic in computer vision research. Depthwise onvolution (DWConv) is widely used in efficient CNNs or Vi…
Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding
Jinze Li, Yixing Xu, Haiduo Huang +4
Speculative decoding (SPD) aims to accelerate the auto-regressive token generation process of a target Large Language Model (LLM). Some approaches employ a draft model with multipl…
Týr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization
Guanchen Li, Yixing Xu, Zeping Li +4
Structural pruning enhances hardware-agnostic inference efficiency for large language models (LLMs) yet often fails to maintain comparable performance. Local pruning performs effic…
Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE
Haiduo Huang, Fuwei Yang, Zhenhua Liu +7
Speculative decoding (SD) accelerates large language model inference by using a smaller draft model to predict multiple tokens, which are then verified in parallel by the larger ta…
MSWA: Refining Local Attention with Multi-ScaleWindow Attention
Yixing Xu, Shivank Nag, Dong Li +2
Transformer-based LLMs have achieved exceptional performance across a wide range of NLP tasks. However, the standard self-attention mechanism suffers from quadratic time complexity…