From the 1 of 9 linked papers with an AI index.
9 papers
NELSSA: A GPU-PNM Heterogeneous System for Mixed-Length LLM Serving via Length-based Request Placement
Sookyung Choi, Seungyong Lee, Kangkyu Park +11
The paper introduces NELSSA, a system that combines GPUs with processing‑near‑memory (PNM) accelerators to efficiently serve large language model requests of varying context length…
A CXL Memory Rack for Multi-Turn LLM Serving
Hakbeom Jang, Inho Song, Sam H. Noh +2
Long-context, multi-turn, and agentic LLM workloads increasingly reuse previously processed context, making KV-cache reuse essential for reducing redundant computation. However, th…
StreamDQ: Near-Memory Weight DeQuantization in Custom HBM for Scalable AI Inference Acceleration
Minki Jeong, Daegun Yoon, Soohong Ahn +7
As large language models (LLMs) scale, their memory and computation demands have grown substantially, making weight-only quantization a widely adopted technique for reducing model…
ITME: Inference Tiered Memory Expansion with Disaggregated CXL-Hybrid Memories
Hakbeom Jang, Younghoon Min, Sunwoong Kim +5
The rapid shift toward agentic and long-context workloads in Large Language Models (LLMs) is pushing the industry beyond the capacity of individual servers toward disaggregated sha…
AI+HW 2035: Shaping the Next Decade
Deming Chen, Jason Cong, Azalia Mirhoseini +27
Artificial intelligence (AI) and hardware (HW) are advancing at unprecedented rates, yet their trajectories have become inseparably intertwined. The global research community lacks…
TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
Dongha Yoon, Younghoon Min, Hoshik Kim +2
Disaggregated LLM serving improves resource efficiency by separating the compute-intensive prefill phase from the latency-critical decode phase. However, this architecture introduc…