Cascaded Semantic and Positional Self-Attention Network for Document Classification

Juyong Jiang, Jie Zhang, Kai Zhang

Transformers have shown great success in learning representations for language modelling. However, an open challenge still remains on how to systematically aggregate semantic infor…