895 citations · 2.1k across the 169 of their papers we have counts for
20 papers · 1 filter
Extending Machine Language Models toward Human-Level Language Understanding
James L. McClelland, Felix Hill, Maja Rudolph +2
Language is crucial for human intelligence, but what exactly is its role? We take language to be a part of a system for understanding and communicating about situations. The human…
Morphological Segmentation Inside-Out
Ryan Cotterell, Arun Kumar, Hinrich Schütze
Morphological segmentation has traditionally been modeled with non-hierarchical models, which yield flat segmentations as output. In many cases, however, proper morphological analy…
Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity
Nina Poerner, Ulli Waltinger, Hinrich Schütze
We address the task of unsupervised Semantic Textual Similarity (STS) by ensembling diverse pre-trained sentence encoders into sentence meta-embeddings. We apply, extend and evalua…
E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT
Nina Poerner, Ulli Waltinger, Hinrich Schütze
We present a novel way of injecting factual knowledge about entities into the pretrained BERT model (Devlin et al., 2019): We align Wikipedia2Vec entity vectors (Yamada et al., 201…
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly
Nora Kassner, Hinrich Schütze
Building on Petroni et al. (2019), we propose two new probing tasks analyzing factual knowledge stored in Pretrained Language Models (PLMs). (1) Negation. We find that PLMs do not…
Linguistically Informed Relation Extraction and Neural Architectures for Nested Named Entity Recognition in BioNLP-OST 2019
Usama Yaseen, Pankaj Gupta, Hinrich Schütze
Named Entity Recognition (NER) and Relation Extraction (RE) are essential tools in distilling knowledge from biomedical literature. This paper presents our findings from participat…
BioNLP-OST 2019 RDoC Tasks: Multi-grain Neural Relevance Ranking Using Topics and Attention Based Query-Document-Sentence Interactions
Yatin Chaudhary, Pankaj Gupta, Hinrich Schütze
This paper presents our system details and results of participation in the RDoC Tasks of BioNLP-OST 2019. Research Domain Criteria (RDoC) construct is a multi-dimensional and broad…
BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance
Timo Schick, Hinrich Schütze
Pretraining deep language models has led to large performance gains in NLP. Despite this success, Schick and Schütze (2020) recently showed that these models struggle to understand…
Type-aware Convolutional Neural Networks for Slot Filling
Heike Adel, Hinrich Schütze
The slot filling task aims at extracting answers for queries about entities from text, such as "Who founded Apple". In this paper, we focus on the relation classification component…
Multi-view and Multi-source Transfers in Neural Topic Modeling with Pretrained Topic and Word Embeddings
Pankaj Gupta, Yatin Chaudhary, Hinrich Schütze
Though word embeddings and topics are complementary representations, several past works have only used pre-trained word embeddings in (neural) topic modeling to address data sparsi…
Neural Architectures for Fine-Grained Propaganda Detection in News
Pankaj Gupta, Khushbu Saxena, Usama Yaseen +2
This paper describes our system (MIC-CIS) details and results of participation in the fine-grained propaganda detection shared task 2019. To address the tasks of sentence (SLC) and…
Morphological Word Embeddings
Ryan Cotterell, Hinrich Schütze
Linguistic similarity is multi-faceted. For instance, two words may be similar with respect to semantics, syntax, or morphology inter alia. Continuous word-embeddings have been sho…
Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings
Yadollah Yaghoobzadeh, Katharina Kann, Timothy J. Hazen +2
Word embeddings typically represent different meanings of a word in a single conflated vector. Empirical analysis of embeddings of ambiguous words is currently limited by the small…
SherLIiC: A Typed Event-Focused Lexical Inference Benchmark for Evaluating Natural Language Inference
Martin Schmitt, Hinrich Schütze
We present SherLIiC, a testbed for lexical inference in context (LIiC), consisting of 3985 manually annotated inference rule candidates (InfCands), accompanied by (i) ~960k unlabel…
A Hierarchical Decoder with Three-level Hierarchical Attention to Generate Abstractive Summaries of Interleaved Texts
Sanjeev Kumar Karn, Francine Chen, Yan-Ying Chen +2
Interleaved texts, where posts belonging to different threads occur in one sequence, are a common occurrence, e.g., online chat conversations. To quickly obtain an overview of such…
Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts
Timo Schick, Hinrich Schütze
Learning high-quality embeddings for rare words is a hard problem because of sparse context information. Mimicking (Pinter et al., 2017) has been proposed as a solution: given embe…
Fine-Grained Argument Unit Recognition and Classification
Dietrich Trautmann, Johannes Daxenberger, Christian Stab +2
Prior work has commonly defined argument retrieval from heterogeneous document collections as a sentence-level classification task. Consequently, argument retrieval suffers both fr…
An Unsupervised Joint System for Text Generation from Knowledge Graphs and Semantic Parsing
Martin Schmitt, Sahand Sharifzadeh, Volker Tresp +1
Knowledge graphs (KGs) can vary greatly from one domain to another. Therefore supervised approaches to both graph-to-text generation and text-to-graph knowledge extraction (semanti…
Analytical Methods for Interpretable Ultradense Word Embeddings
Philipp Dufter, Hinrich Schütze
Word embeddings are useful for a wide variety of tasks, but they lack interpretability. By rotating word spaces, interpretable dimensions can be identified while preserving the inf…
Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking
Timo Schick, Hinrich Schütze
Pretraining deep neural network architectures with a language modeling objective has brought large improvements for many natural language processing tasks. Exemplified by BERT, a r…