7 papers
A Shape-Adaptive Architecture with Disaggregated Quantization for Efficient LLM Serving
Cong Guo, Chiyue Wei, Bowen Duan +5
Large language models (LLMs) have become the backbone of modern AI applications, but pose significant challenges for efficient inference. Their autoregressive generation divides ex…
Vortex: Bridging Extreme Compression and Efficient LLM Inference
Haoxuan Shan, Cong Guo, Bowen Duan +6
Extreme compression techniques, including vector quantization (VQ) and input-dependent sparsity, can significantly reduce the memory footprint of large language models (LLMs). Howe…
Optimus: Elastic Decoding for Efficient Diffusion LLM Serving
Chiyue Wei, Cong Guo, Bowen Duan +7
Large language model (LLM) serving is fundamentally limited by inefficient hardware utilization. Autoregressive (AR) decoding underutilizes GPUs due to its strictly sequential exec…
EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecture
Bowen Duan, Cong Guo, Chiyue Wei +8
Large Language Models (LLMs) have achieved impressive performance across diverse domains but remain inefficient during the autoregressive decoding phase. Unlike the prefill stage,…
FractalCloud: A Fractal-Inspired Architecture for Efficient Large-Scale Point Cloud Processing
Yuzhe Fu, Changchun Zhou, Hancheng Ye +6
Three-dimensional (3D) point clouds are increasingly used in applications such as autonomous driving, robotics, and virtual reality (VR). Point-based neural networks (PNNs) have de…
Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks
Chiyue Wei, Bowen Duan, Cong Guo +4
Spiking Neural Networks (SNNs) are gaining attention for their energy efficiency and biological plausibility, utilizing 0-1 activation sparsity through spike-driven computation. Wh…
Transitive Array: An Efficient GEMM Accelerator with Result Reuse
Cong Guo, Chiyue Wei, Jiaming Tang +4
Deep Neural Networks (DNNs) and Large Language Models (LLMs) have revolutionized artificial intelligence, yet their deployment faces significant memory and computational challenges…