ASTRA: Communication-Efficient Acceleration for Multi-Device Transformer Inference

Xiao Liu, Lijun Zhang, Deepak Ganesan +1

Multi-device inference can reduce Transformer latency by parallelizing computation. However, existing methods require high inter-device bandwidth, making them impractical for bandw…