895 citations · 2.1k across the 169 of their papers we have counts for
24 papers · 1 filter
True Few-Shot Learning with Prompts -- A Real-World Perspective
Timo Schick, Hinrich Schütze
Prompt-based approaches are strong at few-shot learning. However, Perez et al. (2021) have recently cast doubt on their performance because they had difficulty getting good results…
Scene Graph Generation for Better Image Captioning?
Maximilian Mozes, Martin Schmitt, Vladimir Golkov +2
We investigate the incorporation of visual relationships into the task of supervised image caption generation by proposing a model that leverages detected objects and auto-generate…
BERT Cannot Align Characters
Antonis Maronikolakis, Philipp Dufter, Hinrich Schütze
In previous work, it has been shown that BERT can adequately align cross-lingual sentences on the word level. Here we investigate whether BERT can also operate as a char-level alig…
Locating Language-Specific Information in Contextualized Embeddings
Sheng Liang, Philipp Dufter, Hinrich Schütze
Multilingual pretrained language models (MPLMs) exhibit multilinguality and are well suited for transfer across languages. Most MPLMs are trained in an unsupervised fashion and the…
Graph Algorithms for Multiparallel Word Alignment
Ayyoob Imani, Masoud Jalili Sabet, Lütfi Kerem Şenel +3
With the advent of end-to-end deep learning approaches in machine translation, interest in word alignments initially decreased; however, they have again become a focus of research…
Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languages
Antonis Maronikolakis, Philipp Dufter, Hinrich Schütze
The size of the vocabulary is a central design choice in large pretrained language models, with respect to both performance and memory requirements. Typically, subword tokenization…
Continuous Entailment Patterns for Lexical Inference in Context
Martin Schmitt, Hinrich Schütze
Combining a pretrained language model (PLM) with textual patterns has been shown to help in both zero- and few-shot settings. For zero-shot performance, it makes sense to design pa…
Discrete and Soft Prompting for Multilingual Models
Mengjie Zhao, Hinrich Schütze
It has been shown for English that discrete and soft prompting perform strongly in few-shot learning with pretrained language models (PLMs). In this paper, we show that discrete an…
Semi-Automated Labeling of Requirement Datasets for Relation Extraction
Jeremias Bohn, Jannik Fischbach, Martin Schmitt +2
Creating datasets manually by human annotators is a laborious task that can lead to biased and inhomogeneous labels. We propose a flexible, semi-automatic framework for labeling da…
ParCourE: A Parallel Corpus Explorer for a Massively Multilingual Corpus
Ayyoob Imani, Masoud Jalili Sabet, Philipp Dufter +2
With more than 7000 languages worldwide, multilingual natural language processing (NLP) is essential both from an academic and commercial perspective. Researching typological prope…
Data Centric Domain Adaptation for Historical Text with OCR Errors
Luisa März, Stefan Schweter, Nina Poerner +2
We propose new methods for in-domain and cross-domain Named Entity Recognition (NER) on historical data for Dutch and French. For the cross-domain case, we address domain shift by…
Multi-source Neural Topic Modeling in Multi-view Embedding Spaces
Pankaj Gupta, Yatin Chaudhary, Hinrich Schütze
Though word embeddings and topics are complementary representations, several past works have only used pretrained word embeddings in (neural) topic modeling to address data sparsit…
Static Embeddings as Efficient Knowledge Bases?
Philipp Dufter, Nora Kassner, Hinrich Schütze
Recent research investigates factual knowledge stored in large pretrained language models (PLMs). Instead of structural knowledge base (KB) queries, masked sentences such as "Paris…
Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity
Valentin Hofmann, Xiaowen Dong, Janet B. Pierrehumbert +1
The increasing polarization of online political discourse calls for computational tools that automatically detect and monitor ideological divides in social media. We introduce a mi…
Enriching a Model's Notion of Belief using a Persistent Memory
Nora Kassner, Oyvind Tafjord, Hinrich Schutze +1
Although pretrained language models (PTLMs) have been shown to contain significant amounts of world knowledge, they can still produce inconsistent answers to questions when probed,…
Generating Datasets with Pretrained Language Models
Timo Schick, Hinrich Schütze
To obtain high-quality sentence embeddings from pretrained language models (PLMs), they must either be augmented with additional pretraining objectives or finetuned on a large set…
Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data
Sanjeev Kumar Karn, Francine Chen, Yan-Ying Chen +2
Interleaved texts, where posts belonging to different threads occur in a sequence, commonly occur in online chat posts, so that it can be time-consuming to quickly obtain an overvi…
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
Timo Schick, Sahana Udupa, Hinrich Schütze
When trained on large, unfiltered crawls from the internet, language models pick up and reproduce all kinds of undesirable biases that can be found in the data: they often generate…
Does He Wink or Does He Nod? A Challenging Benchmark for Evaluating Word Understanding of Language Models
Lutfi Kerem Senel, Hinrich Schütze
Recent progress in pretraining language models on large corpora has resulted in large performance gains on many NLP tasks. These large models acquire linguistic knowledge during pr…
Position Information in Transformers: An Overview
Philipp Dufter, Martin Schmitt, Hinrich Schütze
Transformers are arguably the main workhorse in recent Natural Language Processing research. By definition a Transformer is invariant with respect to reordering of the input. Howev…
Language Models for Lexical Inference in Context
Martin Schmitt, Hinrich Schütze
Lexical inference in context (LIiC) is the task of recognizing textual entailment between two very similar sentences, i.e., sentences that only differ in one expression. It can the…
Measuring and Improving Consistency in Pretrained Language Models
Yanai Elazar, Nora Kassner, Shauli Ravfogel +4
Consistency of a model -- that is, the invariance of its behavior under meaning-preserving alternations in its input -- is a highly desirable property in natural language processin…
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models
Nora Kassner, Philipp Dufter, Hinrich Schütze
Recently, it has been found that monolingual English language models can be used as knowledge bases. Instead of structural knowledge base queries, masked sentences such as "Paris i…
Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words
Valentin Hofmann, Janet B. Pierrehumbert, Hinrich Schütze
How does the input segmentation of pretrained language models (PLMs) affect their interpretations of complex words? We present the first study investigating this question, taking B…