The Illusion of Latent Generalization: Bi-directionality and the Reversal Curse

Julian Coda-Forno, Jane X. Wang, Arslan Chaudhry

The reversal curse describes a failure of autoregressive language models to retrieve a fact in reverse order (e.g., training on ``$A>B$'' but failing on ``$B<A$''). Recent work…