TRACE: Transformer Residual Activation Codebook Encoding for Communication-efficient Distributed Edge Inference

Muhammad Azlan Qazi, Alexandros Iosifidis, Qi Zhang

Distributed Transformer inference across co-located edge devices is fundamentally communication-bound, as exchanging activations between sequence-parallel partitions dominates infe…