Nearly Optimal Attention Coresets

Edo Liberty, Alexandr Andoni, Eldar Kleiner

We consider the problem of estimating the Attention mechanism in small space, and prove the existence of coresets for it of nearly optimal size. Specifically, we show that for any…