SlideDP: Scaling Host-Resident LLM Fine-Tuning Across Multiple GPUs

Ruijia Yang, Shiyuan Lin, Yulong Ao +5

Host-resident layer streaming enables full-parameter LLM fine-tuning beyond GPU memory, but data-parallel ranks compete for shared host resources. Replicated transfers amplify traf…