cxl 1gpu 1heterogeneous computing 1large language model serving 1mixed-length workloads 1processing-near-memory 1
From the 1 of 3 linked papers with an AI index.
3 papers
cs.AR2026
NELSSA: A GPU-PNM Heterogeneous System for Mixed-Length LLM Serving via Length-based Request Placement
Sookyung Choi, Seungyong Lee, Kangkyu Park +11
The paper introduces NELSSA, a system that combines GPUs with processing‑near‑memory (PNM) accelerators to efficiently serve large language model requests of varying context length…
cs.AR2025
HPU: High-Bandwidth Processing Unit for Scalable, Cost-effective LLM Inference via GPU Co-processing
Myunghyun Rhee, Joonseop Sim, Taeyoung Ahn +6
The attention layer, a core component of Transformer-based LLMs, brings out inefficiencies in current GPU systems due to its low operational intensity and the substantial memory re…
cs.LG2025
MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference
Myunghyun Rhee, Sookyung Choi, Euiseok Kim +3
The escalating context length in Large Language Models (LLMs) creates a severe performance bottleneck around the Key-Value (KV) cache, whose memory-bound nature leads to significan…