Transformers trained on proteins can learn to attend to Euclidean distance

Isaac Ellmen, Constantin Schneider, Matthew I. J. Raybould +1

While conventional Transformers generally operate on sequence data, they can be used in conjunction with structure models, typically SE(3)-invariant or equivariant graph neural net…