Feature Resemblance: Towards a Theoretical Understanding of Analogical Reasoning in Transformers

Ruichen Xu, Wenjing Yan, Ying-Jun Angela Zhang

Understanding reasoning in large language models is complicated by evaluations that conflate multiple reasoning types. We isolate analogical reasoning, where a model transfers an a…