1 citations · 1 across the 4 of their papers we have counts for
4 papers
StreamDQ: Near-Memory Weight DeQuantization in Custom HBM for Scalable AI Inference Acceleration
Minki Jeong, Daegun Yoon, Soohong Ahn +7
As large language models (LLMs) scale, their memory and computation demands have grown substantially, making weight-only quantization a widely adopted technique for reducing model…
Accelerating Sparse Matrix-Matrix Multiplication on GPUs with Processing Near HBMs
Shiju Li, Younghoon Min, Hane Yie +5
Sparse General Matrix-Matrix Multiplication (SpGEMM) is a fundamental operation in numerous scientific computing and data analytics applications, often bottlenecked by irregular me…
MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference
Myunghyun Rhee, Sookyung Choi, Euiseok Kim +3
The escalating context length in Large Language Models (LLMs) creates a severe performance bottleneck around the Key-Value (KV) cache, whose memory-bound nature leads to significan…
HPU: High-Bandwidth Processing Unit for Scalable, Cost-effective LLM Inference via GPU Co-processing
Myunghyun Rhee, Joonseop Sim, Taeyoung Ahn +6
The attention layer, a core component of Transformer-based LLMs, brings out inefficiencies in current GPU systems due to its low operational intensity and the substantial memory re…