3 papers
cs.LG2026
Can Transformers Learn to Verify During Backtracking Search?
Yin Jun Phua, Tony Ribeiro, Tuan Nguyen +1
Backtracking search underlies classical constraint solvers, planners, and theorem provers. Recent transformer-based reasoning systems explore search trees over their own intermedia…
cs.LG2024
Transformers Use Causal World Models in Maze-Solving Tasks
Alex F. Spies, William Edwards, Michael I. Ivanitskiy +5
Recent studies in interpretability have explored the inner workings of transformer models trained on tasks across various domains, often discovering that these networks naturally d…
cs.LG2023
Structured World Representations in Maze-Solving Transformers
Michael Igorevich Ivanitskiy, Alex F. Spies, Tilman Räuker +9
Transformer models underpin many recent advances in practical machine learning applications, yet understanding their internal behavior continues to elude researchers. Given the siz…