Learning the greatest common divisor: explaining transformer predictions

François Charton

The predictions of small transformers, trained to calculate the greatest common divisor (GCD) of two positive integers, can be fully characterized by looking at model inputs and ou…