The Cost of Context: Profiling the Energy Footprint of Input Tokens in Large Language Models

Boris Ruf, Marcin Detyniecki

The rapid adoption of Retrieval-Augmented Generation (RAG) and long-context LLMs has shifted the computational burden toward the prefill phase---the initial stage where the model p…