Shrink the longest: improving latent space isotropy with symplicial geometry

Sergei Kudriashov, Olesya Karpik, Eduard Klyshinsky

Although transformer-based models have been dominating the field of deep learning, various studies of their embedding space have shown that they suffer from "representation degener…