WavePP: High-Throughput Pipeline Parallel LLM Prefill under Prefix Reuse

Aaryam Sharma

Pipeline parallelism can improve prefill throughput by processing multiple request chunks concurrently across different stages of the model. However, keeping the pipeline fully uti…