117 citations · 151 across the 8 of their papers we have counts for
4 papers · 1 filter
LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale
Jaehong Cho, Minsu Kim, Hyunmin Choi +2
Recently, there has been an extensive research effort in building efficient large language model (LLM) inference serving systems. These efforts not only include innovations in the…
ONNXim: A Fast, Cycle-level Multi-core NPU Simulator
Hyungkyu Ham, Wonhyuk Yang, Yunseon Shin +5
As DNNs are widely adopted in various application domains while demanding increasingly higher compute and memory requirements, designing efficient and performant NPUs (Neural Proce…
NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing
Guseul Heo, Sangyeop Lee, Jaehong Cho +6
Modern transformer-based Large Language Models (LLMs) are constructed with a series of decoder blocks. Each block comprises three key components: (1) QKV generation, (2) multi-head…
Accelerating String-Key Learned Index Structures via Memoization-based Incremental Training
Minsu Kim, Jinwoo Hwang, Guseul Heo +3
Learned indexes use machine learning models to learn the mappings between keys and their corresponding positions in key-value indexes. These indexes use the mapping information as…