Anchor Attention, Small Cache: Code Generation with Large Language Models

Xiangyu Zhang, Yu Zhou, Guang Yang +2

The development of large language models (LLMs) has revolutionized automated code generation. However, their high demand of computation resources has hindered a broader deployment…