Power Aware Dynamic Reallocation For Inference

Yiwei Jiang, Sangeeta Chowdhary, Nathaniel Morris +3

Disaggregation has emerged as a powerful strategy for optimizing large language model (LLM) inference by separating compute-intensive prefill and memory-bound decode phases across…