76 citations · 148 across the 26 of their papers we have counts for
31 papers
Joint Lemmatization and Morphological Tagging with LEMMING
Thomas Muller, Ryan Cotterell, Alexander Fraser +1
We present LEMMING, a modular log-linear model that jointly models lemmatization and tagging and supports the integration of arbitrary global features. It is trainable on corpora a…
Labeled Morphological Segmentation with Semi-Markov Models
Ryan Cotterell, Thomas Müller, Alexander Fraser +1
We present labeled morphological segmentation, an alternative view of morphological processing that unifies several tasks. From an annotation standpoint, we additionally introduce…
Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity
Katharina Hämmerl, Alina Fastowski, Jindřich Libovický +1
Previous work has shown that the representations output by contextual language models are more anisotropic than static type embeddings, and typically display outlier dimensions. Th…
On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss
Yihong Liu, Alexandra Chronopoulou, Hinrich Schütze +1
Although unsupervised neural machine translation (UNMT) has achieved success in many language pairs, the copying problem, i.e., directly copying some parts of the input sentence as…
How to Solve Few-Shot Abusive Content Detection Using the Data We Actually Have
Viktor Hangya, Alexander Fraser
Due to the broad range of social media platforms, the requirements of abusive language detection systems are varied and ever-changing. Already a large set of annotated corpora with…
Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation
Wen Lai, Alexandra Chronopoulou, Alexander Fraser
Despite advances in multilingual neural machine translation (MNMT), we argue that there are still two major challenges in this area: data imbalance and representation degeneration.…
AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models
Alexandra Chronopoulou, Matthew E. Peters, Alexander Fraser +1
Pretrained language models (PLMs) are trained on massive corpora, but often need to specialize to specific domains. A parameter-efficient adaptation method suggests training an ada…
Speaking Multiple Languages Affects the Moral Bias of Language Models
Katharina Hämmerl, Björn Deiseroth, Patrick Schramowski +4
Pre-trained multilingual language models (PMLMs) are commonly used when dealing with data from multiple languages and cross-lingual transfer. However, PMLMs are trained on varying…
: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter
Wen Lai, Alexandra Chronopoulou, Alexander Fraser
Multilingual neural machine translation models (MNMT) yield state-of-the-art performance when evaluated on data from a domain and language pair seen at training time. However, when…
A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing
Sophie Henning, William Beluch, Alexander Fraser +1
Many natural language processing (NLP) tasks are naturally imbalanced, as some target categories occur much more frequently than others in the real world. In such scenarios, curren…
Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation
Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser
Large multilingual models trained with self-supervision achieve state-of-the-art results in a wide range of natural language processing tasks. Self-supervised pretrained models are…
Don't Forget Cheap Training Signals Before Building Unsupervised Bilingual Word Embeddings
Silvia Severini, Viktor Hangya, Masoud Jalili Sabet +2
Bilingual Word Embeddings (BWEs) are one of the cornerstones of cross-lingual transfer of NLP models. They can be built using only monolingual corpora without supervision leading t…
Demonstrating CAT: Synthesizing Data-Aware Conversational Agents for Transactional Databases
Marius Gassen, Benjamin Hättasch, Benjamin Hilprecht +3
Databases for OLTP are often the backbone for applications such as hotel room or cinema ticket booking applications. However, developing a conversational agent (i.e., a chatbot-lik…
Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategies
Marion Weller-Di Marco, Matthias Huck, Alexander Fraser
Morphologically rich languages pose difficulties to machine translation. Machine translation engines that rely on statistical learning from parallel training data, such as state-of…
Do Multilingual Language Models Capture Differing Moral Norms?
Katharina Hämmerl, Björn Deiseroth, Patrick Schramowski +3
Massively multilingual sentence representations are trained on large corpora of uncurated data, with a very imbalanced proportion of languages included in the training. This may ca…
Combining Static and Contextualised Multilingual Embeddings
Katharina Hämmerl, Jindřich Libovický, Alexander Fraser
Static and contextual multilingual embeddings have complementary strengths. Static embeddings, while less expressive than contextual language models, can be more straightforwardly…
Addressing the Challenges of Cross-Lingual Hate Speech Detection
Irina Bigoulaeva, Viktor Hangya, Iryna Gurevych +1
The goal of hate speech detection is to filter negative online content aiming at certain groups of people. Due to the easy accessibility of social media platforms it is crucial to…
Improving Both Domain Robustness and Domain Adaptability in Machine Translation
Wen Lai, Jindřich Libovický, Alexander Fraser
We consider two problems of NMT domain adaptation using meta-learning. First, we want to reach domain robustness, i.e., we want to reach high quality on both domains seen in the tr…
Why don't people use character-level machine translation?
Jindřich Libovický, Helmut Schmid, Alexander Fraser
We present a literature and empirical survey that critically assesses the state of the art in character-level modeling for machine translation (MT). Despite evidence in the literat…
Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation
Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser
Successful methods for unsupervised neural machine translation (UNMT) employ crosslingual pretraining via self-supervision, often in the form of a masked language modeling or a seq…
Neural String Edit Distance
Jindřich Libovický, Alexander Fraser
We propose the neural string edit distance model for string-pair matching and string transduction based on learnable string edit distance. We modify the original expectation-maximi…
The LMU Munich System for the WMT 2020 Unsupervised Machine Translation Shared Task
Alexandra Chronopoulou, Dario Stojanovski, Viktor Hangya +1
This paper describes the submission of LMU Munich to the WMT 2020 unsupervised shared task, in two language directions, German<->Upper Sorbian. Our core unsupervised neural machine…
Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT
Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser
Using a language model (LM) pretrained on two languages with large monolingual data in order to initialize an unsupervised neural machine translation (UNMT) system yields state-of-…
Anchor-based Bilingual Word Embeddings for Low-Resource Languages
Tobias Eder, Viktor Hangya, Alexander Fraser
Good quality monolingual word embeddings (MWEs) can be built for languages which have large amounts of unlabeled text. MWEs can be aligned to bilingual spaces using only a few thou…