Probing for Arithmetic Errors in Language Models

Yucheng Sun, Alessandro Stolfo, Mrinmaya Sachan

We investigate whether internal activations in language models can be used to detect arithmetic errors. Starting with a controlled setting of 3-digit addition, we show that simple…