On the Intrinsic Limits of Transformer Image Embeddings in Non-Solvable Spatial Reasoning

Siyi Lyu, Quan Liu, Feng Yan

Vision Transformers (ViTs) excel in semantic recognition but exhibit systematic failures in spatial reasoning tasks such as mental rotation. While often attributed to data scale, t…