Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping

Muru Zhang, Mayank Mishra, Zhongzhu Zhou +7

Large language model inference is both memory-intensive and time-consuming, often requiring distributed algorithms to efficiently scale. Various model parallelism strategies are us…