1 citations · 1 across the 4 of their papers we have counts for
1 paper · 1 filter
Leshan Li, Zhui Zhu, Xianglong Deng +6
Auto-regressive model decode is bandwidth-bound, since every weight and key/value-cache byte crosses high-bandwidth memory once per token. A megakernel is an ideal solution, but ex…