Energy-Efficient Multimodal Inference Serving with Tri-serve

Ziyang Jia, Sara Rashidi Golrouye, Laxmi Bhuyan +5

Multimodal model inference creates substantial energy demand with growing performance requirements. Within GPUs, power is autonomously managed by an on-board power management unit…