Does Entity Abstraction Help Generative Transformers Reason?

Nicolas Gontier, Siva Reddy, Christopher Pal

We study the utility of incorporating entity type abstractions into pre-trained Transformers and test these methods on four NLP tasks requiring different forms of logical reasoning…