An Inquiry into Datacenter TCO for LLM Inference with FP8

Jiwoo Kim, Joonhyung Lee, Gunho Park +4

As large language models (LLMs) continue to scale, the high power consumption of AI accelerators in datacenters presents significant challenges, substantially increasing the total…