HELIX: HEterogeneous LLM Inference with FleXible and Fine-Grained Partitioning

Victoria Clerico, William Simon, Corey Liam Lammie +6

LLM inference is inherently heterogeneous, with performance bottlenecks shifting throughout inference phases down to individual operators. This has motivated heterogeneous serving…