When Models Don't Manipulate Manifolds: The Geometry of a Comparison Task

Sai Sumedh R. Hindupur, Hadas Orgad, Thomas Fel +1

One of the current premises of mechanistic interpretability research is that detailed accounts of the geometry of neural network representations can tell us how models perform comp…