TokenPowerBench: Benchmarking the Power Consumption of LLM Inference

Chenxu Niu, Wei Zhang, Jie Li +4

Large language model (LLM) services now answer billions of queries per day, and industry reports show that inference, not training, accounts for more than 90% of total power consum…