Comparison of different Unique hard attention transformer models by the formal languages they can recognize

Leonid Ryvkin

This note is a survey of various results on the capabilities of unique hard attention transformers encoders (UHATs) to recognize formal languages. We distinguish between masked vs.…