Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Rethinking the Role of Positional Encoding: Sliding-Window Transformers without PE Remain Turing Complete
Qian Li, Xinyu Mao, Shang-Hua Teng
Positional encoding (PE) is widely viewed as necessary for transformers to process ordered sequences: without them, the next-token map appears permutation-invariant in its context…
cs.LG2026
On the Expressive Power and Limitations of Multi-Layer SSMs
Nikola Zubić, Qian Li, Yuyi Wang +1
We study how depth, finite precision, state dimension, and chain-of-thought (CoT) affect the expressive power of multi-layer state-space models (SSMs). For the explicit-table -f…