Deriving Contextualised Semantic Features from BERT (and Other Transformer Model) Embeddings

Jacob Turton, David Vinson, Robert Elliott Smith

Models based on the transformer architecture, such as BERT, have marked a crucial step forward in the field of Natural Language Processing. Importantly, they allow the creation of…