SpecInF: Exploiting Idle GPU Resources in Distributed DL Training via Speculative Inference Filling

Cunchi Lv, Xiao Shi, Dong Liang +2

Deep Learning (DL), especially with Large Language Models (LLMs), brings benefits to various areas. However, DL training systems usually yield prominent idling GPU resources due to…