14 papers
A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference
Zhuoran Song, Haozhe Jiang, Chunyu Qi +4
Vision-Language-Action (VLA) models have demonstrated strong potential for embodied AI, yet their high inference latency on GPUs limits real-time deployment. Existing accelerators,…
COSM: A Cooperative Scheduling Framework for Concurrent PIM and CPU Execution on Mobile Devices
Yilong Zhao, Fangxin Liu, Onur Mutlu +4
The development of on-device large language models (LLMs) is driven by the need for privacy and fast response times. Energy-intensive data transfer on mobile devices makes Processi…
Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference
Xuwen Zhou, Fangxin Liu, Chao Wang +5
Speculative decoding accelerates autoregressive generation by letting draft tokens bypass full verification, but conventional frameworks suffer from frequent false rejections, part…
SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization
Zhixiong Zhao, Fangxin Liu, Junjie Wang +4
The emergence of accurate open large language models (LLMs) has sparked a push for advanced quantization techniques to enable efficient deployment on end-user devices. In this pape…
GEMM-GS: Accelerating 3D Gaussian Splatting on Tensor Cores with GEMM-Compatible Blending
Haomin Li, Bowen Zhu, Fangxin Liu +4
Neural Radiance Fields (NeRF) enables 3D scene reconstruction from several 2D images but incurs high rendering latency via its point-sampling design. 3D Gaussian Splatting (3DGS) i…
QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operations
Zhixiong Zhao, Haomin Li, Fangxin Liu +5
Transformer-based models have revolutionized computer vision (CV) and natural language processing (NLP) by achieving state-of-the-art performance across a range of benchmarks. Howe…