Toward Robust and Efficient ML-Based GPU Caching for Modern Inference

Peng Chen, Jiaji Zhang, Hailiang Zhao +11

In modern GPU inference, cache efficiency remains a major bottleneck, and heuristic policies such as \textsc{LRU} can perform far worse than the offline optimum. Existing learning-…