1 paper
Vaibhav Singh, Zafir Khalid, Pietro Cagnasso +2
Pre-training large neural networks at scale imposes heavy memory demands on accelerators and often requires costly communication. We introduce Subnetwork Data Parallelism (SDP), a…