Automatic Model-Hardware Co-Adaptation for Heterogeneous AI Accelerators

Tian Chen, Mingheng Mi, Pu Wang +2

Large language models now evolve faster than production inference systems can be ported and optimized. New releases change attention, MoE routing, quantization formats, KV-cache la…