1 citations · 1 across the 2 of their papers we have counts for
8 papers
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification
Timo Schick, Helmut Schmid, Hinrich Schütze
A recent approach for few-shot text classification is to convert textual inputs to cloze questions that contain some form of task description, process them with a pretrained langua…
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
Timo Schick, Hinrich Schütze
When scaled to hundreds of billions of parameters, pretrained language models such as GPT-3 (Brown et al., 2020) achieve remarkable few-shot performance. However, enormous amounts…
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference
Timo Schick, Hinrich Schütze
Some NLP tasks can be solved in a fully unsupervised fashion by providing a pretrained language model with "task descriptions" in natural language (e.g., Radford et al., 2019). Whi…
BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance
Timo Schick, Hinrich Schütze
Pretraining deep language models has led to large performance gains in NLP. Despite this success, Schick and Schütze (2020) recently showed that these models struggle to understand…
Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts
Timo Schick, Hinrich Schütze
Learning high-quality embeddings for rare words is a hard problem because of sparse context information. Mimicking (Pinter et al., 2017) has been proposed as a solution: given embe…
Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking
Timo Schick, Hinrich Schütze
Pretraining deep neural network architectures with a language modeling objective has brought large improvements for many natural language processing tasks. Exemplified by BERT, a r…