10Cache: Heterogeneous Resource-Aware Tensor Caching and Migration for LLM Training

Sabiha Afroz, Redwan Ibne Seraj Khan, Hadeel Albahar +2

Training large language models (LLMs) in the cloud faces growing memory bottlenecks due to the limited capacity and high cost of GPUs. While GPU memory offloading to CPU and NVMe h…