Geometric Structure Predicts Per-Example Neighborhood Loss Under PCA Truncation

Sriram Loganathan, Gokul Anand, Ragavan Arivazhagan

Dimensionality reduction can preserve the average performance of text embeddings even when a substantial fraction of their dimensions is removed. Aggregate metrics, however, do not…