MatKV: Trading Compute for Flash Storage in LLM Inference

Kun-Woo Shin, Jay H. Park, Moonwook Oh +3

We observe two major trends in LLM-based generative AI: (1) inference is becoming the dominant factor in terms of cost and power consumption, surpassing training, and (2) retrieval…