895 citations · 2.1k across the 169 of their papers we have counts for
205 papers
The Boundaries of Automation: A Theory of Persistent Human Participation
Fares Fourati, Hinrich Schütze, Eyke Hüllermeier +1
The rapid progress of AI has intensified the long-standing pursuit of automation: replacing human participation with algorithms wherever possible. Implicit in this pursuit is the a…
Behavioral and Representational Evidence of Binomial Ordering Preferences in Large Language Models
Zhiqing Yang, Yilun Liu, Yunpu Ma +2
Large language models (LLMs) can readily reproduce conventional expressions, yet their ability to model gradient frequency distributions remains underexplored. We investigate this…
Output Vector Editing for Memorization Mitigation in Large Language Models
Ahmad Dawar Hakimi, Kaiwei Lei, Isabelle Augenstein +1
Large language models can reproduce training text, creating privacy, copyright, and security risks. We propose Output Vector Editing (OVE), a post-hoc method for suppressing text c…
Edit the Bits, Diff the Codes: Bitwise Residual Editing for Visual Autoregressive Models
Shengqiang Zhang, Ruotong Liao, Volker Tresp +2
Text-guided image editing with visual autoregressive (VAR) generators requires controlling both what the model samples and where the sampled change is written back into the image c…
Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation?
Renhao Pei, Yihong Liu, Sampo Pyysalo +2
Large language models (LLMs) offer a promising approach to machine translation (MT) for extremely low-resource languages by incorporating linguistic resources through in-context le…
Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models
Yuetian Lu, Ali Modarressi, Yihong Liu +1
Activation patching can identify a mixture-of-experts (MoE) block whose clean output restores a corrupted factual prediction. However, because the block output combines contributio…
ChunkFT: Byte-Streamed Optimization for Memory-Efficient Full Fine-Tuning
Yongkang Liu, Zijing Wang, Mengjie Zhao +7
This work presents \textsc{ChunkFT}, a memory-efficient fine-tuning framework that reformulates full-parameter fine-tuning around a dynamically activated working set. \textsc{Chunk…
SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
Yongkang Liu, Xing Li, Mengjie Zhao +7
As the number of model parameters increases, parameter-efficient fine-tuning (PEFT) has become the go-to choice for tailoring pre-trained large language models. Low-rank Adaptation…
GKnow: Measuring the Entanglement of Gender Bias and Factual Gender
Leonor Veloso, Hinrich Schütze
Recent works have analyzed the impact of individual components of neural networks on gendered predictions, often with a focus on mitigating gender bias. However, mechanistic interp…
Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders
Yana Veitsman, Yihong Liu, Hinrich Schütze
Cross-lingual alignment is often assumed to improve cross-lingual transfer by bringing representations of different languages closer together. However, improvements in representati…
Why Do More Experts Fail? A Theoretical Analysis of Model Merging
Zijing Wang, Xingle Xu, Yongkang Liu +6
Model merging dramatically reduces storage and computational resources by combining multiple expert models into a single multi-task model. Although recent model merging methods hav…
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
Enes Özeren, Yihong Liu, Hinrich Schütze
Many pre-trained language models (PLMs) exhibit suboptimal performance on mid- and low-resource languages, largely due to limited exposure to these languages during pre-training. A…
Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu
Renhao Pei, Yihong Liu, Peiqin Lin +2
In-context machine translation (MT) with large language models (LLMs) is a promising approach for low-resource MT, as it can readily take advantage of linguistic resources such as…
A Federated Approach to Few-Shot Hate Speech Detection for Marginalized Communities
Haotian Ye, Axel Wisiorek, Antonis Maronikolakis +2
Hate speech online remains an understudied issue for marginalized communities, particularly in the Global South, which includes developing societies with increasing internet penetr…
EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models
Shaoxiong Ji, Zihao Li, Jaakko Paavola +7
In this work, we introduce EMMA-500, a large-scale multilingual language model continue-trained on texts across 546 languages designed for enhanced multilingual performance, focusi…
ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Language
Yongkang Liu, Feng Shi, Daling Wang +2
Although large language models(LLMs) show amazing capabilities, among various exciting applications discovered for LLMs fall short in other low-resource languages. Besides, most ex…
Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts
Chunlan Ma, Yihong Liu, Haotian Ye +1
Decoder-only large language models (LLMs) excel in high-resource languages across various tasks through few-shot or even zero-shot in-context learning (ICL). However, their perform…
A Unified Data Augmentation Framework for Low-Resource Multi-Domain Dialogue Generation
Yongkang Liu, Ercong Nie, Shi Feng +5
Current state-of-the-art dialogue systems heavily rely on extensive training datasets. However, challenges arise in domains where domain-specific training datasets are insufficient…
Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment
Orgest Xhelili, Yihong Liu, Hinrich Schütze
Multilingual pre-trained models (mPLMs) have shown impressive performance on cross-lingual transfer tasks. However, the transfer performance is often hindered when a low-resource t…
BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
Ercong Nie, Bo Shao, Zifeng Ding +3
This paper introduces BMIKE-53, a comprehensive benchmark for cross-lingual in-context knowledge editing (IKE) across 53 languages, unifying three knowledge editing (KE) datasets:…
Joint Lemmatization and Morphological Tagging with LEMMING
Thomas Muller, Ryan Cotterell, Alexander Fraser +1
We present LEMMING, a modular log-linear model that jointly models lemmatization and tagging and supports the integration of arbitrary global features. It is trainable on corpora a…
TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data
Yihong Liu, Chunlan Ma, Haotian Ye +1
Transliterating related languages that use different scripts into a common script is effective for improving crosslingual transfer in downstream tasks. However, this methodology of…
XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples
Peiqin Lin, André F. T. Martins, Hinrich Schütze
Recent studies indicate that leveraging off-the-shelf or fine-tuned retrievers, capable of retrieving relevant in-context examples tailored to the input query, enhances few-shot in…
Labeled Morphological Segmentation with Semi-Markov Models
Ryan Cotterell, Thomas Müller, Alexander Fraser +1
We present labeled morphological segmentation, an alternative view of morphological processing that unifies several tasks. From an annotation standpoint, we additionally introduce…