11 papers
Optimus: Elastic Decoding for Efficient Diffusion LLM Serving
Chiyue Wei, Cong Guo, Bowen Duan +7
Large language model (LLM) serving is fundamentally limited by inefficient hardware utilization. Autoregressive (AR) decoding underutilizes GPUs due to its strictly sequential exec…
EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecture
Bowen Duan, Cong Guo, Chiyue Wei +8
Large Language Models (LLMs) have achieved impressive performance across diverse domains but remain inefficient during the autoregressive decoding phase. Unlike the prefill stage,…
MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM
Dongwei Wang, Jinhee Kim, Seokho Han +8
Dynamic runtime latency and memory constraints necessitate flexible large language model (LLM) deployment, where an LLM can be inferred with various quantization precisions based o…
Focus: A Streaming Concentration Architecture for Efficient Vision-Language Models
Chiyue Wei, Cong Guo, Junyao Zhang +8
Vision-Language Models (VLMs) have demonstrated strong performance on tasks such as video captioning and visual question answering. However, their growing scale and video-level inp…
FractalCloud: A Fractal-Inspired Architecture for Efficient Large-Scale Point Cloud Processing
Yuzhe Fu, Changchun Zhou, Hancheng Ye +6
Three-dimensional (3D) point clouds are increasingly used in applications such as autonomous driving, robotics, and virtual reality (VR). Point-based neural networks (PNNs) have de…
Platinum: Path-Adaptable LUT-Based Accelerator Tailored for Low-Bit Weight Matrix Multiplication
Haoxuan Shan, Cong Guo, Chiyue Wei +4
The rapid scaling of large language models demands more efficient hardware. Quantization offers a promising trade-off between efficiency and performance. With ultra-low-bit quantiz…
CAMformer: Associative Memory is All You Need
Tergel Molom-Ochir, Benjamin F. Morris, Mark Horton +8
Transformers face scalability challenges due to the quadratic cost of attention, which involves dense similarity computations between queries and keys. We propose CAMformer, a nove…
DPad: Efficient Diffusion Language Models with Suffix Dropout
Xinhua Chen, Sitao Huang, Cong Guo +5
Diffusion-based Large Language Models (dLLMs) parallelize text generation by framing decoding as a denoising process, but suffer from high computational overhead since they predict…
Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks
Chiyue Wei, Bowen Duan, Cong Guo +4
Spiking Neural Networks (SNNs) are gaining attention for their energy efficiency and biological plausibility, utilizing 0-1 activation sparsity through spike-driven computation. Wh…
AutoRAC: Automated Processing-in-Memory Accelerator Design for Recommender Systems
Feng Cheng, Tunhou Zhang, Junyao Zhang +6
The performance bottleneck of deep-learning-based recommender systems resides in their backbone Deep Neural Networks. By integrating Processing-In-Memory~(PIM) architectures, resea…
Ecco: Improving Memory Bandwidth and Capacity for LLMs via Entropy-aware Cache Compression
Feng Cheng, Cong Guo, Chiyue Wei +7
Large language models (LLMs) have demonstrated transformative capabilities across diverse artificial intelligence applications, yet their deployment is hindered by substantial memo…