A Logic for Expressing Log-Precision Transformers

William Merrill, Ashish Sabharwal

One way to interpret the reasoning power of transformer-based language models is to describe the types of logical rules they can resolve over some input text. Recently, Chiang et a…