895 citations · 2.1k across the 169 of their papers we have counts for
189 papers · 1 filter
Behavioral and Representational Evidence of Binomial Ordering Preferences in Large Language Models
Zhiqing Yang, Yilun Liu, Yunpu Ma +2
Large language models (LLMs) can readily reproduce conventional expressions, yet their ability to model gradient frequency distributions remains underexplored. We investigate this…
Output Vector Editing for Memorization Mitigation in Large Language Models
Ahmad Dawar Hakimi, Kaiwei Lei, Isabelle Augenstein +1
Large language models can reproduce training text, creating privacy, copyright, and security risks. We propose Output Vector Editing (OVE), a post-hoc method for suppressing text c…
Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation?
Renhao Pei, Yihong Liu, Sampo Pyysalo +2
Large language models (LLMs) offer a promising approach to machine translation (MT) for extremely low-resource languages by incorporating linguistic resources through in-context le…
Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models
Yuetian Lu, Ali Modarressi, Yihong Liu +1
Activation patching can identify a mixture-of-experts (MoE) block whose clean output restores a corrupted factual prediction. However, because the block output combines contributio…
GKnow: Measuring the Entanglement of Gender Bias and Factual Gender
Leonor Veloso, Hinrich Schütze
Recent works have analyzed the impact of individual components of neural networks on gendered predictions, often with a focus on mitigating gender bias. However, mechanistic interp…
Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders
Yana Veitsman, Yihong Liu, Hinrich Schütze
Cross-lingual alignment is often assumed to improve cross-lingual transfer by bringing representations of different languages closer together. However, improvements in representati…
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
Enes Özeren, Yihong Liu, Hinrich Schütze
Many pre-trained language models (PLMs) exhibit suboptimal performance on mid- and low-resource languages, largely due to limited exposure to these languages during pre-training. A…
Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu
Renhao Pei, Yihong Liu, Peiqin Lin +2
In-context machine translation (MT) with large language models (LLMs) is a promising approach for low-resource MT, as it can readily take advantage of linguistic resources such as…
A Federated Approach to Few-Shot Hate Speech Detection for Marginalized Communities
Haotian Ye, Axel Wisiorek, Antonis Maronikolakis +2
Hate speech online remains an understudied issue for marginalized communities, particularly in the Global South, which includes developing societies with increasing internet penetr…
EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models
Shaoxiong Ji, Zihao Li, Jaakko Paavola +7
In this work, we introduce EMMA-500, a large-scale multilingual language model continue-trained on texts across 546 languages designed for enhanced multilingual performance, focusi…
ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Language
Yongkang Liu, Feng Shi, Daling Wang +2
Although large language models(LLMs) show amazing capabilities, among various exciting applications discovered for LLMs fall short in other low-resource languages. Besides, most ex…
Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts
Chunlan Ma, Yihong Liu, Haotian Ye +1
Decoder-only large language models (LLMs) excel in high-resource languages across various tasks through few-shot or even zero-shot in-context learning (ICL). However, their perform…
A Unified Data Augmentation Framework for Low-Resource Multi-Domain Dialogue Generation
Yongkang Liu, Ercong Nie, Shi Feng +5
Current state-of-the-art dialogue systems heavily rely on extensive training datasets. However, challenges arise in domains where domain-specific training datasets are insufficient…
Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment
Orgest Xhelili, Yihong Liu, Hinrich Schütze
Multilingual pre-trained models (mPLMs) have shown impressive performance on cross-lingual transfer tasks. However, the transfer performance is often hindered when a low-resource t…
BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
Ercong Nie, Bo Shao, Zifeng Ding +3
This paper introduces BMIKE-53, a comprehensive benchmark for cross-lingual in-context knowledge editing (IKE) across 53 languages, unifying three knowledge editing (KE) datasets:…
Joint Lemmatization and Morphological Tagging with LEMMING
Thomas Muller, Ryan Cotterell, Alexander Fraser +1
We present LEMMING, a modular log-linear model that jointly models lemmatization and tagging and supports the integration of arbitrary global features. It is trainable on corpora a…
TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data
Yihong Liu, Chunlan Ma, Haotian Ye +1
Transliterating related languages that use different scripts into a common script is effective for improving crosslingual transfer in downstream tasks. However, this methodology of…
XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples
Peiqin Lin, André F. T. Martins, Hinrich Schütze
Recent studies indicate that leveraging off-the-shelf or fine-tuned retrievers, capable of retrieving relevant in-context examples tailored to the input query, enhances few-shot in…
Labeled Morphological Segmentation with Semi-Markov Models
Ryan Cotterell, Thomas Müller, Alexander Fraser +1
We present labeled morphological segmentation, an alternative view of morphological processing that unifies several tasks. From an annotation standpoint, we additionally introduce…
GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network
Shuzhou Yuan, Ercong Nie, Michael Färber +2
Large Language Models (LLMs) exhibit strong In-Context Learning (ICL) capabilities when prompts with demonstrations are used. However, fine-tuning still remains crucial to further…
ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks
Bolei Ma, Ercong Nie, Shuzhou Yuan +4
Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding. However, most previous studies primarily f…
MaLA-500: Massive Language Adaptation of Large Language Models
Peiqin Lin, Shaoxiong Ji, Jörg Tiedemann +2
Large language models (LLMs) have advanced the state of the art in natural language processing. However, their predominant design for English or a limited set of languages creates…
TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models
Yihong Liu, Chunlan Ma, Haotian Ye +1
The world's more than 7000 languages are written in at least 293 scripts. Due to various reasons, many closely related languages use different scripts, which poses a difficulty for…
MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer
Haotian Ye, Yihong Liu, Chunlan Ma +1
Transformer-based pre-trained language models (PLMs) have achieved remarkable performance in various natural language processing (NLP) tasks. However, pre-training such models can…