InferOpt: Constrained Multi-Objective Search for LLM Inference Configurations

Qi Chen, Yingying Cheng, Zhaoyi Sun +3

Serving an LLM means setting dozens of inference-time knobs, from per-layer KV retention to per-layer expert counts. Practice sets them with mechanism-specific heuristics that retu…