3 papers
cs.AR2025
Scalable Processing-Near-Memory for 1M-Token LLM Inference: CXL-Enabled KV-Cache Management Beyond GPU Limits
Dowon Kim, MinJae Lee, Janghyeon Kim +8
The expansion of context windows in large language models (LLMs) to multi-million tokens introduces severe memory and compute bottlenecks, particularly in managing the growing Key-…
cs.AR2025
Cosmos: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search
Seoyoung Ko, Hyunjeong Shim, Wanju Doh +8
Retrieval-Augmented Generation (RAG) is crucial for improving the quality of large language models by injecting proper contexts extracted from external sources. RAG requires high-t…
cs.OS2025
Adaptive Migration Decision for Multi-Tenant Memory Systems
Hyungjun Cho, Igjae Kim, Kwanghoon Choi +5
Tiered memory systems consisting of fast small memory and slow large memory have emerged to provide high capacity memory in a cost-effective way. The effectiveness of tiered memory…