Longer Attention Span: Increasing Transformer Context Length with Sparse Graph Processing Techniques

Nathaniel Tomczak, Sanmukh Kuppannagari

Transformers have demonstrated great success in numerous domains including natural language processing and bioinformatics. This success stems from the use of the attention mechanis…