4 citations · 10 across the 5 of their papers we have counts for
1 paper · 1 filter
Rajmohan Rajaraman, Ravi Sundaram, Amanuel Tesfaye
What can a single layer of self-attention compute? We study head complexity: the minimum number of attention heads required to compute a function in a one-layer attention-only mode…