PruneX: A Hierarchical Communication-Efficient System for Distributed CNN Training with Structured Pruning

Alireza Olama, Andreas Lundell, Izzat El Hajj +2

Inter-node communication bandwidth increasingly constrains distributed training at scale on multi-node GPU clusters. While compact models are the ultimate deployment target, conven…