Uncovering Graph Reasoning in Decoder-only Transformers with Circuit Tracing

Xinnan Dai, Chung-Hsiang Lo, Kai Guo +3

Transformer-based LLMs demonstrate strong performance on graph reasoning tasks, yet their internal mechanisms remain underexplored. To uncover these reasoning process mechanisms in…