SEIS: Self-Evolving Inference Systems

Zhen Xu, Jingyu Liu, Zongze Li +2

Inference systems determine how fast and how cheaply language models can be served, so making them faster has direct practical value. However, prior work focuses mostly on optimizi…