Dual Triangle Attention: Effective Bidirectional Attention Without Positional Embeddings

Logan Hallee, Jason P. Gleghorn

Bidirectional transformers are the foundation of many sequence modeling tasks across natural, biological, and chemical language domains, but they are permutation-invariant without…