9 papers
OASIS: Outlier-Aware LUT-Based GEMM with Dual-Side Quantization for LLM Inference Acceleration
Xueying Wu, Baijun Zhou, Zhihui Gao +4
Large language models (LLMs) have demonstrated impressive capabilities across a wide range of applications, but demand substantial memory and compute resources during inference. Ex…
EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecture
Bowen Duan, Cong Guo, Chiyue Wei +8
Large Language Models (LLMs) have achieved impressive performance across diverse domains but remain inefficient during the autoregressive decoding phase. Unlike the prefill stage,…
DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
Zishan Shao, Lixun Zhang, Kangning Cui +10
Large language models (LLMs) handle many tasks with one set of parameters, but under KV-cached inference it is unclear what task-general structure, if any, is used at decode time r…
FlashFPS: Efficient Farthest Point Sampling for Large-Scale Point Clouds via Pruning and Caching
Yuzhe Fu, Hancheng Ye, Cong Guo +7
Point-based Neural Networks (PNNs) have become a key approach for point cloud processing. However, a core operation in these models, Farthest Point Sampling (FPS), often introduces…
HippoMM: Hippocampal-inspired Multimodal Memory for Long Audiovisual Event Understanding
Yueqian Lin, Jingyang Zhang, Qinsi Wang +5
Comprehending extended audiovisual experiences remains challenging for computational systems, particularly temporal integration and cross-modal associations fundamental to human ep…
KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
Hancheng Ye, Zhengqi Gao, Mingyuan Ma +8
Multi-agent large language model (LLM) systems are increasingly adopted for complex language processing tasks that require communication and coordination among agents. However, the…