Relational reasoning and inductive bias in transformers and large language models

Jesse Geerts, Andrew Liu, Stephanie Chan +2

Transformer-based models have demonstrated remarkable reasoning abilities, but the mechanisms underlying relational reasoning remain poorly understood. We investigate how transform…