FlashVector: Agent for Hierarchical Model Serving Stack Optimization

Qi Wu, Lohan Lemire, Kai Meng +8

Model serving is one of the largest cost drivers in production recommender systems. Maximizing its throughput requires navigating a deeply layered hierarchy: GPU kernels, the ML fr…