Chiron: Cache Management Empowered by LLM Intelligence for AI Workloads

Bohan Zhang, Renshu Ge, Tianze Wang +1

AI clusters keep data in remote object stores and rely on local caches to stop GPUs from waiting for data. A cache performs well only when its prefetching and eviction policies mat…