TRANSIT: Transparent Scale-in for Multi-Node LLM Training

Hyungyo Kim, Nicholas Satchanov, Hrishi Shah +9

TRANSIT is a transparent scale-in framework to enable multi-node model training on fewer GPUs while maintaining training efficiency by transparently leveraging CPU DRAM as an exten…