1 citations · 1 across the 8 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Disentangling the Expressivity of RoPE
Selim Jerad, Anej Svete, Jiaoda Li +1
Two accounts recur in explanations of the success of rotary position embeddings (RoPE). Expressivity studies associate periodic position information with modular predicates, wherea…
cs.CL2026
Characterizing the Expressivity of Local Attention in Transformers
Jiaoda Li, Ryan Cotterell
The transformer is the most popular neural architecture for language modeling. The cornerstone of the transformer is its global attention mechanism, which lets the model aggregate…