Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms

Prabhu Vellaisamy, Vanessa Lam, Shawn Blanton +1

Large language model (LLM) inference serving is priced by tokens, but GPU energy is consumed over inference windows. This accounting mismatch makes token-normalized metrics incompl…