2 papers
cs.LG2025
On the Limitations and Capabilities of Position Embeddings for Length Generalization
Yang Chen, Yitao Liang, Zhouchen Lin
In Transformers, Position Embeddings (PEs) significantly influence Length Generalization (LG) performance, yet their fundamental role remains unclear. In this work, we investigate…
cs.LG2025
Low-Dimension-to-High-Dimension Generalization And Its Implications for Length Generalization
Yang Chen, Long Yang, Yitao Liang +1
Low-Dimension-to-High-Dimension (LDHD) generalization is a special case of Out-of-Distribution (OOD) generalization, where the training data are restricted to a low-dimensional sub…