13 papers
Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference
Xuwen Zhou, Fangxin Liu, Chao Wang +5
Speculative decoding accelerates autoregressive generation by letting draft tokens bypass full verification, but conventional frameworks suffer from frequent false rejections, part…
SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization
Zhixiong Zhao, Fangxin Liu, Junjie Wang +4
The emergence of accurate open large language models (LLMs) has sparked a push for advanced quantization techniques to enable efficient deployment on end-user devices. In this pape…
GEMM-GS: Accelerating 3D Gaussian Splatting on Tensor Cores with GEMM-Compatible Blending
Haomin Li, Bowen Zhu, Fangxin Liu +4
Neural Radiance Fields (NeRF) enables 3D scene reconstruction from several 2D images but incurs high rendering latency via its point-sampling design. 3D Gaussian Splatting (3DGS) i…
QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operations
Zhixiong Zhao, Haomin Li, Fangxin Liu +5
Transformer-based models have revolutionized computer vision (CV) and natural language processing (NLP) by achieving state-of-the-art performance across a range of benchmarks. Howe…
LaMoS: Enabling Efficient Large Number Modular Multiplication through SRAM-based CiM Acceleration
Haomin Li, Fangxin Liu, Chenyang Guan +3
Barrett's algorithm is one of the most widely used methods for performing modular multiplication, a critical nonlinear operation in modern privacy computing techniques such as homo…
FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization
Fangxin Liu, Zongwu Wang, JinHong Xia +6
The rapid advancement of large language models (LLMs) has exacerbated the memory bottleneck due to the widening gap between model parameter scaling and hardware capabilities. While…