Simulating Weighted Automata over Sequences and Trees with Transformers

Michael Rizvi, Maude Lizaire, Clara Lacroce +1

Transformers are ubiquitous models in the natural language processing (NLP) community and have shown impressive empirical successes in the past few years. However, little is unders…