CritiPrefill: A Segment-wise Criticality-based Approach for Prefilling Acceleration in LLMs

Junlin Lv, Yuan Feng, Xike Xie +3

Large language models have achieved notable success across various domains, yet efficient inference is still limited by the quadratic computation complexity of the attention mechan…