3 papers
cs.LG2026
Three Tokens Force Exponential Feature Rank in Nonnegative Kernel Attention
Vicente Opazo
Full attention exposes every token pair, whereas kernel attention compresses a sequence into a fixed-dimensional sketch. We show that this distinction becomes exponential at the fi…
cs.LG2026
Indexing: the Beginning and the End
Alexander Kozachinskiy, Vicente Opazo, Felipe Urrutia
We study information bottlenecks in modern deep-learning architectures -- RNNs, softmax transformers, linear-attention transformers and state-space models -- through the lens of th…
cs.CC2026
Polynomial-Time Mistake-Bounded Language Generation
Héctor Jimenez, Alexander Kozachinskiy, Vicente Opazo
In this note, we introduce a polynomial-time version of the mistake-bounded language generation (MBLG) framework due to Kleinberg, Peale, and Reingold (2026). We observe that the f…