83 citations · 536 across the 64 of their papers we have counts for
11 papers · 2 filters
Sparse Interventions in Language Models with Differentiable Masking
Nicola De Cao, Leon Schmid, Dieuwke Hupkes +1
There has been a lot of interest in understanding what information is captured by hidden representations of language models (LMs). Typically, interpretation methods i) do not guara…
Learning Opinion Summarizers by Selecting Informative Reviews
Arthur Bražinskas, Mirella Lapata, Ivan Titov
Opinion summarization has been traditionally approached with unsupervised, weakly-supervised and few-shot learning techniques. In this work, we collect a large dataset of summaries…
Highly Parallel Autoregressive Entity Linking with Discriminative Correction
Nicola De Cao, Wilker Aziz, Ivan Titov
Generative approaches have been recently shown to be effective for both Entity Disambiguation and Entity Linking (i.e., joint mention detection and disambiguation). However, the pr…
Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT
Elena Voita, Rico Sennrich, Ivan Titov
Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural ne…
Meta-Learning to Compositionally Generalize
Henry Conklin, Bailin Wang, Kenny Smith +1
Natural language is compositional; the meaning of a sentence is a function of the meaning of its parts. This property allows humans to create and interpret novel sentences, general…
Exploring Unsupervised Pretraining Objectives for Machine Translation
Christos Baziotis, Ivan Titov, Alexandra Birch +1
Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT), by drastically reducing the need for large parallel data. Most approaches ad…
Structured Reordering for Modeling Latent Alignments in Sequence Transduction
Bailin Wang, Mirella Lapata, Ivan Titov
Despite success in many domains, neural models struggle in settings where train and test examples are drawn from different distributions. In particular, in contrast to humans, conv…
Learning from Executions for Semantic Parsing
Bailin Wang, Mirella Lapata, Ivan Titov
Semantic parsing aims at translating natural language (NL) utterances onto machine-interpretable programs, which can be executed against a real-world environment. The expensive ann…
Editing Factual Knowledge in Language Models
Nicola De Cao, Wilker Aziz, Ivan Titov
The factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual infe…
Sparse Attention with Linear Units
Biao Zhang, Ivan Titov, Rico Sennrich
Recently, it has been argued that encoder-decoder models can be made more interpretable by replacing the softmax function in the attention with its sparse variants. In this work, w…
An Empirical Study of Compound PCFGs
Yanpeng Zhao, Ivan Titov
Compound probabilistic context-free grammars (C-PCFGs) have recently established a new state of the art for unsupervised phrase-structure grammar induction. However, due to the hig…