71 citations · 217 across the 26 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
RePo: Language Models with Context Re-Positioning
Huayang Li, Tianyu Zhao, Deng Cai +1
In-context learning is fundamental to modern Large Language Models (LLMs); however, prevailing architectures impose a rigid and fixed contextual structure by assigning linear or co…
cs.LG2025
SeqPE: Transformer with Sequential Position Encoding
Huayang Li, Yahui Liu, Hongyu Sun +5
Since self-attention layers in Transformers are permutation invariant by design, positional encodings must be explicitly incorporated to enable spatial understanding. However, fixe…