Token Homogenization under Positional Bias

Viacheslav Yusupov, Danil Maksimov, Ameliia Alaeva +9

This paper investigates token homogenization - the convergence of token representations toward uniformity across transformer layers and its relationship to positional bias in large…