5 papers · 1 filter
Cornserve: A Distributed Serving System for Any-to-Any Multimodal Models
Jae-Won Chung, Jeff J. Ma, Jisang Ahn +4
Any-to-Any models are an emerging class of multimodal models that accept combinations of multimodal data (e.g., text, image, video, audio) as input and generate them as output. Ser…
Where Do the Joules Go? Diagnosing Inference Energy Consumption
Jae-Won Chung, Ruofan Wu, Jeff J. Ma +1
Energy is now a critical ML computing resource. While measuring energy consumption and observing trends is a valuable first step, accurately understanding and diagnosing why those…
Cornfigurator: Automated Planning for Any-to-Any Multimodal Model Serving
Jeff J. Ma, Jae-Won Chung, Jisang Ahn +5
Any-to-Any models are an emerging class of multimodal models that accept combinations of text and multimodal data as input and generate them as output, introducing heterogeneous co…
TetriServe: Efficiently Serving Mixed DiT Workloads
Runyu Lu, Shiqi He, Wenxuan Tan +5
Diffusion Transformer (DiT) models excel at generating high-quality images through iterative denoising steps, but serving them under strict Service Level Objectives (SLOs) is chall…
The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization
Jae-Won Chung, Jeff J. Ma, Ruofan Wu +5
As the adoption of Generative AI in real-world services grow explosively, energy has emerged as a critical bottleneck resource. However, energy remains a metric that is often overl…