212 citations · 741 across the 81 of their papers we have counts for
17 papers · 1 filter
A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English
Marius Mosbach, Stefania Degaetano-Ortlieb, Marie-Pauline Krielke +2
Transformer-based language models achieve high performance on various tasks, but we still lack understanding of the kind of linguistic knowledge they learn and rely on. We evaluate…
Rediscovering the Slavic Continuum in Representations Emerging from Neural Models of Spoken Language Identification
Badr M. Abdullah, Jacek Kudera, Tania Avgustinova +2
Deep neural networks have been employed for various spoken language recognition tasks, including tasks that are multilingual by definition such as spoken language identification. I…
Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages
Michael A. Hedderich, David Adelani, Dawei Zhu +3
Multilingual transformer models like mBERT and XLM-RoBERTa have obtained great improvements for many NLP tasks on a variety of languages. However, recent works also showed that res…
On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers
Marius Mosbach, Anna Khokhlova, Michael A. Hedderich +1
Fine-tuning pre-trained contextualized embedding models has become an integral part of the NLP pipeline. At the same time, probing has emerged as a way to investigate the linguisti…
HUMAN: Hierarchical Universal Modular ANnotator
Moritz Wolf, Dana Ruiter, Ashwin Geet D'Sa +3
A lot of real-world phenomena are complex and cannot be captured by single task annotations. This causes a need for subsequent annotations, with interdependent questions and answer…
Fusion Models for Improved Visual Captioning
Marimuthu Kalimuthu, Aditya Mogadala, Marius Mosbach +1
Visual captioning aims to generate textual descriptions given images or videos. Traditionally, image captioning models are trained on human annotated datasets such as Flickr30k and…
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios
Michael A. Hedderich, Lukas Lange, Heike Adel +2
Deep neural networks and huge language models are becoming omnipresent in natural language applications. As they are known for requiring large amounts of training data, there is a…
FAME: Feature-Based Adversarial Meta-Embeddings for Robust Input Representations
Lukas Lange, Heike Adel, Jannik Strötgen +1
Combining several embeddings typically improves performance in downstream tasks as different embeddings encode different information. It has been shown that even models using embed…
Cross-Domain Adaptation of Spoken Language Identification for Related Languages: The Curious Case of Slavic Languages
Badr M. Abdullah, Tania Avgustinova, Bernd Möbius +1
State-of-the-art spoken language identification (LID) systems, which are based on end-to-end deep neural networks, have shown remarkable success not only in discriminating between…
Privacy Guarantees for De-identifying Text Transformations
David Ifeoluwa Adelani, Ali Davody, Thomas Kleinbauer +1
Machine Learning approaches to Natural Language Processing tasks benefit from a comprehensive collection of real-life user data. At the same time, there is a clear need for protect…
Integrating Image Captioning with Rule-based Entity Masking
Aditya Mogadala, Xiaoyu Shen, Dietrich Klakow
Given an image, generating its natural language description (i.e., caption) is a well studied problem. Approaches proposed to address this problem usually rely on image features th…
Sparse Graph to Sequence Learning for Vision Conditioned Long Textual Sequence Generation
Aditya Mogadala, Marius Mosbach, Dietrich Klakow
Generating longer textual sequences when conditioned on the visual information is an interesting problem to explore. The challenge here proliferate over the standard vision conditi…
On the effect of normalization layers on Differentially Private training of deep Neural networks
Ali Davody, David Ifeoluwa Adelani, Thomas Kleinbauer +1
Differentially private stochastic gradient descent (DPSGD) is a variation of stochastic gradient descent based on the Differential Privacy (DP) paradigm, which can mitigate privacy…
Learning Functions to Study the Benefit of Multitask Learning
Gabriele Bettgenhäuser, Michael A. Hedderich, Dietrich Klakow
We study and quantify the generalization patterns of multitask learning (MTL) models for sequence labeling tasks. MTL models are trained to optimize a set of related tasks jointly.…
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
Marius Mosbach, Maksym Andriushchenko, Dietrich Klakow
Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks. Despite the strong empiri…
Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence
Xiaoyu Shen, Ernie Chang, Hui Su +2
The neural attention model has achieved great success in data-to-text generation tasks. Though usually excelling at producing fluent text, it suffers from the problem of informatio…
Distant Supervision and Noisy Label Learning for Low Resource Named Entity Recognition: A Study on Hausa and Yorùbá
David Ifeoluwa Adelani, Michael A. Hedderich, Dawei Zhu +2
The lack of labeled training data has limited the development of natural language processing tools, such as named entity recognition, for many languages spoken in developing countr…