cache hierarchy 1emerging memory 1energy efficiency 1large language models 1monolithic 3d integration 1
From the 1 of 2 linked papers with an AI index.
2 papers
cs.AR2026
LLMET: Enabling Cross-Layer Evaluation of Emerging M3D Memories for Energy-Efficient LLM Serving
Ming-Yen Lee, Hanchen Yang, Faaiq Waqar +4
The paper introduces LLMET, a cross‑layer simulation framework that evaluates how emerging monolithic 3D (M3D) on‑chip memory can cut energy use when serving large language models,…
cs.AR2025
Architecting Long-Context LLM Acceleration with Packing-Prefetch Scheduler and Ultra-Large Capacity On-Chip Memories
Ming-Yen Lee, Faaiq Waqar, Hanchen Yang +3
Long-context Large Language Model (LLM) inference faces increasing compute bottlenecks as attention calculations scale with context length, primarily due to the growing KV-cache tr…