8 citations · 11 across the 4 of their papers we have counts for
Showing 2019Show all
3 papers · 1 filter
cs.CL2019
BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance
Timo Schick, Hinrich Schütze
Pretraining deep language models has led to large performance gains in NLP. Despite this success, Schick and Schütze (2020) recently showed that these models struggle to understand…
cs.CL2019
Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts
Timo Schick, Hinrich Schütze
Learning high-quality embeddings for rare words is a hard problem because of sparse context information. Mimicking (Pinter et al., 2017) has been proposed as a solution: given embe…
cs.CL2019
Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking
Timo Schick, Hinrich Schütze
Pretraining deep neural network architectures with a language modeling objective has brought large improvements for many natural language processing tasks. Exemplified by BERT, a r…