LeetDecoding: A PyTorch Library for Exponentially Decaying Causal Linear Attention with CUDA Implementations

Jiaping Wang, Simiao Zhang, Qiao-Chu He +1

The machine learning and data science community has made significant while dispersive progress in accelerating transformer-based large language models (LLMs), and one promising app…