Transformers Can Learn Connectivity in Some Graphs but Not Others

Amit Roy, Abulhair Saparov

Reasoning capability is essential to ensure the factual correctness of the responses of transformer-based Large Language Models (LLMs), and robust reasoning about transitive relati…