Extracting Moore Machines from Transformers using Queries and Counterexamples

Rik Adriaensen, Jaron Maene

Fuelled by the popularity of the transformer architecture in deep learning, several works have investigated what formal languages a transformer can learn from data. Nonetheless, ex…