Examining Two Hop Reasoning Through Information Content Scaling

David Johnston, Nora Belrose

Prior work has found that transformers have an inconsistent ability to learn to answer latent two-hop questions -- questions of the form "Who is Bob's mother's boss?" We study why…