Parallel Track Transformers: Enabling Fast GPU Inference with Reduced Synchronization

Chong Wang, Nan Du, Tom Gunter +8

Efficient large-scale inference of transformer-based large language models (LLMs) remains a fundamental systems challenge, frequently requiring multi-GPU parallelism to meet string…