A Training-free Sub-quadratic Cost Transformer Model Serving Framework With Hierarchically Pruned Attention

Heejun Lee, Geon Park, Youngwan Lee +7

In modern large language models (LLMs), increasing the context length is crucial for improving comprehension and coherence in long-context, multi-modal, and retrieval-augmented lan…