Distributed Training using an Intelligent Network

Nihar Shah, Ben Blier

Distributed training across a wide area network (WAN) is challenging, as continuous parameter exchange by islands of compute is constrained by limited bandwidth, high latency, and…