5 citations · 5 across the 2 of their papers we have counts for
6 papers
Static Embeddings as Efficient Knowledge Bases?
Philipp Dufter, Nora Kassner, Hinrich Schütze
Recent research investigates factual knowledge stored in large pretrained language models (PLMs). Instead of structural knowledge base (KB) queries, masked sentences such as "Paris…
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models
Nora Kassner, Philipp Dufter, Hinrich Schütze
Recently, it has been found that monolingual English language models can be used as knowledge bases. Instead of structural knowledge base queries, masked sentences such as "Paris i…
Dirichlet-Smoothed Word Embeddings for Low-Resource Settings
Jakob Jungmaier, Nora Kassner, Benjamin Roth
Nowadays, classical count-based word embeddings using positive pointwise mutual information (PPMI) weighted co-occurrence matrices have been widely superseded by machine-learning-b…
Are Pretrained Language Models Symbolic Reasoners Over Knowledge?
Nora Kassner, Benno Krojer, Hinrich Schütze
How can pretrained language models (PLMs) learn factual knowledge from the training set? We investigate the two most important mechanisms: reasoning and memorization. Prior work ha…
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA
Nora Kassner, Hinrich Schütze
Khandelwal et al. (2020) use a k-nearest-neighbor (kNN) component to improve language model performance. We show that this idea is beneficial for open-domain question answering (QA…
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly
Nora Kassner, Hinrich Schütze
Building on Petroni et al. (2019), we propose two new probing tasks analyzing factual knowledge stored in Pretrained Language Models (PLMs). (1) Negation. We find that PLMs do not…