Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge

Pengxiao Lin, Zheng-An Chen, Zhi-Qin John Xu

Large Language Models (LLMs) excel at multi-hop reasoning in distribution, yet fail on unseen compositions, a phenomenon known as the curse of two-hop reasoning. In this work, we a…