895 citations · 2.1k across the 169 of their papers we have counts for
28 papers · 1 filter
Multilingual Word Embeddings for Low-Resource Languages using Anchors and a Chain of Related Languages
Viktor Hangya, Silvia Severini, Radoslav Ralev +2
Very low-resource languages, having only a few million tokens worth of data, are not well-supported by multilingual NLP approaches due to poor quality cross-lingual word representa…
OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining
Yihong Liu, Peiqin Lin, Mingyang Wang +1
Instead of pretraining multilingual language models from scratch, a more efficient method is to adapt existing pretrained language models (PLMs) to new languages via vocabulary ext…
Counting the Bugs in ChatGPT's Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model
Leonie Weissweiler, Valentin Hofmann, Anjali Kantharuban +10
Large language models (LLMs) have recently reached an impressive level of linguistic capability, prompting comparisons with human language skills. However, there have been relative…
GradSim: Gradient-Based Language Grouping for Effective Multilingual Training
Mingyang Wang, Heike Adel, Lukas Lange +2
Most languages of the world pose low-resource challenges to natural language processing models. With multilingual training, knowledge can be shared among languages. However, not al…
Unleashing the Multilingual Encoder Potential: Boosting Zero-Shot Performance via Probability Calibration
Ercong Nie, Helmut Schmid, Hinrich Schütze
Pretrained multilingual encoder models can directly perform zero-shot multilingual tasks or linguistic probing by reformulating the input examples into cloze-style prompts. This is…
GlotLID: Language Identification for Low-Resource Languages
Amir Hossein Kargaran, Ayyoob Imani, François Yvon +1
Several recent papers have published good solutions for language identification (LID) for about 300 high-resource and medium-resource languages. However, there is no LID available…
GlotScript: A Resource and Tool for Low Resource Writing System Identification
Amir Hossein Kargaran, François Yvon, Hinrich Schütze
We present GlotScript, an open resource and tool for low resource writing system identification. GlotScript-R is a resource that provides the attested writing systems for more than…
Cross-Lingual Constituency Parsing for Middle High German: A Delexicalized Approach
Ercong Nie, Helmut Schmid, Hinrich Schütze
Constituency parsing plays a fundamental role in advancing natural language processing (NLP) tasks. However, training an automatic syntactic analysis system for ancient languages s…
Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understanding
Bolei Ma, Ercong Nie, Helmut Schmid +1
Multilingual pretrained language models (MPLMs) have demonstrated substantial performance improvements in zero-shot cross-lingual transfer across various natural language understan…
Politeness Stereotypes and Attack Vectors: Gender Stereotypes in Japanese and Korean Language Models
Victor Steinborn, Antonis Maronikolakis, Hinrich Schütze
In efforts to keep up with the rapid progress and use of large language models, gender bias research is becoming more prevalent in NLP. Non-English bias research, however, is still…
On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss
Yihong Liu, Alexandra Chronopoulou, Hinrich Schütze +1
Although unsupervised neural machine translation (UNMT) has achieved success in many language pairs, the copying problem, i.e., directly copying some parts of the input sentence as…
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages
Ayyoob Imani, Peiqin Lin, Amir Hossein Kargaran +8
The NLP community has mainly focused on scaling Large Language Models (LLMs) vertically, i.e., making them better for about 100 languages. We instead scale LLMs horizontally: we cr…
A Crosslingual Investigation of Conceptualization in 1335 Languages
Yihong Liu, Haotian Ye, Leonie Weissweiler +4
Languages differ in how they divide up the world into concepts and words; e.g., in contrast to English, Swahili has a single concept for `belly' and `womb'. We investigate these di…
A study of conceptual language similarity: comparison and evaluation
Haotian Ye, Yihong Liu, Hinrich Schütze
An interesting line of research in natural language processing (NLP) aims to incorporate linguistic typology to bridge linguistic diversity and assist the research of low-resource…
Sociocultural knowledge is needed for selection of shots in hate speech detection tasks
Antonis Maronikolakis, Abdullatif Köksal, Hinrich Schütze
We introduce HATELEXICON, a lexicon of slurs and targets of hate speech for the countries of Brazil, Germany, India and Kenya, to aid training and interpretability of models. We de…
Evaluate What You Can't Evaluate: Unassessable Quality for Generated Response
Yongkang Liu, Shi Feng, Daling Wang +2
LLMs (large language models) such as ChatGPT have shown remarkable language understanding and generation capabilities. Although reference-free evaluators based on LLMs show better…
RET-LLM: Towards a General Read-Write Memory for Large Language Models
Ali Modarressi, Ayyoob Imani, Mohsen Fayyaz +1
Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) through their extensive parameters and comprehensive data utilization. Howev…
Language Models with Rationality
Nora Kassner, Oyvind Tafjord, Ashish Sabharwal +3
While large language models (LLMs) are proficient at question-answering (QA), it is not always clear how (or even if) an answer follows from their latent "beliefs". This lack of in…
mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models
Peiqin Lin, Chengzhi Hu, Zheyu Zhang +2
Recent multilingual pretrained language models (mPLMs) have been shown to encode strong language-specific signals, which are not explicitly provided during pretraining. It remains…
Language-Agnostic Bias Detection in Language Models with Bias Probing
Abdullatif Köksal, Omer Faruk Yalcin, Ahmet Akbiyik +3
Pretrained language models (PLMs) are key components in NLP, but they contain strong social biases. Quantifying these biases is challenging because current methods focusing on fill…
Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs
Yihong Liu, Haotian Ye, Leonie Weissweiler +2
In comparative linguistics, colexification refers to the phenomenon of a lexical form conveying two or more distinct meanings. Existing work on colexification patterns relies on an…
Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages
Chunlan Ma, Ayyoob ImaniGooghari, Haotian Ye +3
While natural language processing tools have been developed extensively for some of the world's languages, a significant portion of the world's over 7000 languages are still neglec…
NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis
Mingyang Wang, Heike Adel, Lukas Lange +2
This paper describes our system developed for the SemEval-2023 Task 12 "Sentiment Analysis for Low-resource African Languages using Twitter Dataset". Sentiment analysis is one of t…
Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages
Verena Blaschke, Hinrich Schütze, Barbara Plank
One of the challenges with finetuning pretrained language models (PLMs) is that their tokenizer is optimized for the language(s) it was pretrained on, but brittle when it comes to…