4 papers · 1 filter
Where Do the Joules Go? Diagnosing Inference Energy Consumption
Jae-Won Chung, Ruofan Wu, Jeff J. Ma +1
Energy is now a critical ML computing resource. While measuring energy consumption and observing trends is a valuable first step, accurately understanding and diagnosing why those…
Kareus: Joint Reduction of Dynamic and Static Energy in Large Model Training
Ruofan Wu, Jae-Won Chung, Mosharaf Chowdhury
The computing demand of AI is growing at an unprecedented rate, but energy supply is not keeping pace. As a result, energy has become an expensive and contended resource that requi…
TetriServe: Efficiently Serving Mixed DiT Workloads
Runyu Lu, Shiqi He, Wenxuan Tan +5
Diffusion Transformer (DiT) models excel at generating high-quality images through iterative denoising steps, but serving them under strict Service Level Objectives (SLOs) is chall…
The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization
Jae-Won Chung, Jeff J. Ma, Ruofan Wu +5
As the adoption of Generative AI in real-world services grow explosively, energy has emerged as a critical bottleneck resource. However, energy remains a metric that is often overl…