7 papers
SpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruning
Hanzhen Wang, Jiaming Xu, Yushun Xiang +4
Pruning is a typical acceleration technique for compute-bound models by removing computation on unimportant values. Recently, it has been applied to accelerate Vision-Language-Acti…
DynSplit-KV: Dynamic Semantic Splitting for KVCache Compression in Efficient Long-Context LLM Inference
Jiancai Ye, Jun Liu, Qingchen Li +5
Although Key-Value (KV) Cache is essential for efficient large language models (LLMs) inference, its growing memory footprint in long-context scenarios poses a significant bottlene…
SpeContext: Enabling Efficient Long-context Reasoning with Speculative Context Sparsity in LLMs
Jiaming Xu, Jiayi Pan, Hanzhen Wang +4
In this paper, we point out that the objective of the retrieval algorithms is to align with the LLM, which is similar to the objective of knowledge distillation in LLMs. We analyze…
SpecDiff: Accelerating Diffusion Model Inference with Self-Speculation
Jiayi Pan, Jiaming Xu, Yongkang Zhou +1
Feature caching has recently emerged as a promising method for diffusion model acceleration. It effectively alleviates the inefficiency problem caused by high computational require…
BalanceGS: Algorithm-System Co-design for Efficient 3D Gaussian Splatting Training on GPU
Junyi Wu, Jiaming Xu, Jinhao Li +4
3D Gaussian Splatting (3DGS) has emerged as a promising 3D reconstruction technique. The traditional 3DGS training pipeline follows three sequential steps: Gaussian densification,…
Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective
Jinhao Li, Jiaming Xu, Shan Huang +9
Large Language Models (LLMs) have demonstrated remarkable capabilities across various fields, from natural language understanding to text generation. Compared to non-generative LLM…