Power law attention biases for molecular transformers

Jay Shen, Yifeng Tang, Andrew Ferguson

Transformers are the go-to architecture for most data modalities due to their scalability. While they have been applied extensively to molecular property prediction, they do not do…