Pipeline Parallelism with Controllable Memory

Penghui Qi, Xinyi Wan, Nyamdavaa Amar +1

Pipeline parallelism has been widely explored, but most existing schedules lack a systematic methodology. In this paper, we propose a framework to decompose pipeline schedules as r…