565 citations · 818 across the 51 of their papers we have counts for
4 papers · 2 filters
The Curious Case of Absolute Position Embeddings
Koustuv Sinha, Amirhossein Kazemnejad, Siva Reddy +3
Transformer language models encode the notion of word order using positional information. Most commonly, this positional information is represented by absolute position embeddings…
Text Characterization Toolkit
Daniel Simig, Tianlu Wang, Verna Dankers +4
In NLP, models are usually evaluated by reporting single-number performance scores on a number of readily available benchmarks, without much deeper analysis. Here, we argue that -…
State-of-the-art generalisation research in NLP: A taxonomy and review
Dieuwke Hupkes, Mario Giulianelli, Verna Dankers +17
The ability to generalise well is one of the primary desiderata of natural language processing (NLP). Yet, what 'good generalisation' entails and how it should be evaluated is not…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…