Publications (21)
Hateful Person or Hateful Model? Investigating the Role of Personas in Hate Speech Detection by Large Language Models
Shuzhou Yuan, Ercong Nie, Mario Tawfelis +3
Hate speech detection is a socially sensitive and inherently subjective task, with judgments often varying based on personal traits. While prior work has examined how socio-demogra…
ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks
Bolei Ma, Ercong Nie, Shuzhou Yuan +4
Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding. However, most previous studies primarily f…
XToM: Exploring the Multilingual Theory of Mind for Large Language Models
Chunkit Chan, Yauwai Yim, Hongchuan Zeng +14
Theory of Mind (ToM), the ability to infer mental states in others, is pivotal for human social cognition. Existing evaluations of ToM in LLMs are largely limited to English, negle…
Cross-Lingual Constituency Parsing for Middle High German: A Delexicalized Approach
Ercong Nie, Helmut Schmid, Hinrich Schütze
Constituency parsing plays a fundamental role in advancing natural language processing (NLP) tasks. However, training an automatic syntactic analysis system for ancient languages s…
Large Language Models as Neurolinguistic Subjects: Discrepancy between Performance and Competence
Linyang He, Ercong Nie, Helmut Schmid +3
This study investigates the linguistic understanding of Large Language Models (LLMs) regarding signifier (form) and signified (meaning) by distinguishing two LLM assessment paradig…
Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understanding
Bolei Ma, Ercong Nie, Helmut Schmid +1
Multilingual pretrained language models (MPLMs) have demonstrated substantial performance improvements in zero-shot cross-lingual transfer across various natural language understan…
LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxification
Shuzhou Yuan, Ercong Nie, Lukas Kouba +4
Detoxification, the task of rewriting harmful language into non-toxic text, has become increasingly important amid the growing prevalence of toxic content online. However, high-qua…
EXECUTE: A Multilingual Benchmark for LLM Token Understanding
Lukas Edman, Helmut Schmid, Alexander Fraser
The CUTE benchmark showed that LLMs struggle with character understanding in English. We extend it to more languages with diverse scripts and writing systems, introducing EXECUTE.…
Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages
Ercong Nie, Sheng Liang, Helmut Schmid +1
Multilingual Pretrained Language Models (MPLMs) have shown their strong multilinguality in recent empirical cross-lingual transfer studies. In this paper, we propose the Prompts Au…
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification
Timo Schick, Helmut Schmid, Hinrich Schütze
A recent approach for few-shot text classification is to convert textual inputs to cloze questions that contain some form of task description, process them with a pretrained langua…
Unleashing the Multilingual Encoder Potential: Boosting Zero-Shot Performance via Probability Calibration
Ercong Nie, Helmut Schmid, Hinrich Schütze
Pretrained multilingual encoder models can directly perform zero-shot multilingual tasks or linguistic probing by reformulating the input examples into cloze-style prompts. This is…
GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network
Shuzhou Yuan, Ercong Nie, Michael Färber +2
Large Language Models (LLMs) exhibit strong In-Context Learning (ICL) capabilities when prompts with demonstrations are used. However, fine-tuning still remains crucial to further…
Why don't people use character-level machine translation?
JindÅich Libovický, Helmut Schmid, Alexander Fraser
We present a literature and empirical survey that critically assesses the state of the art in character-level modeling for machine translation (MT). Despite evidence in the literat…
Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models
Ercong Nie, Helmut Schmid, Hinrich Schütze
Language confusion -- where large language models (LLMs) generate unintended languages against the user's need -- remains a critical challenge, especially for English-centric model…
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages
Ayyoob Imani, Peiqin Lin, Amir Hossein Kargaran +8
The NLP community has mainly focused on scaling Large Language Models (LLMs) vertically, i.e., making them better for about 100 languages. We instead scale LLMs horizontally: we cr…
CUTE: Measuring LLMs' Understanding of Their Tokens
Lukas Edman, Helmut Schmid, Alexander Fraser
Large Language Models (LLMs) show remarkable performance on a wide variety of tasks. Most LLMs split text into multi-character tokens and process them as atomic units without direc…
BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
Ercong Nie, Bo Shao, Zifeng Ding +3
This paper introduces BMIKE-53, a comprehensive benchmark for cross-lingual in-context knowledge editing (IKE) across 53 languages, unifying three knowledge editing (KE) datasets:…
Language Model Re-rankers are Fooled by Lexical Similarities
Lovisa Hagström, Ercong Nie, Ruben Halifa +3
Language model (LM) re-rankers are used to refine retrieval results for retrieval-augmented generation (RAG). They are more expensive than lexical matching methods like BM25 but as…
Part-of-Speech Tagging with Neural Networks
Helmut Schmid
Text corpora which are tagged with part-of-speech information are useful in many areas of linguistic research. In this paper, a new part-of-speech tagging method based on neural ne…
XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairs
Linyang He, Ercong Nie, Sukru Samet Dindar +10
We introduce XCOMPS in this work, a multilingual conceptual minimal pair dataset covering 17 languages. Using this dataset, we evaluate LLMs' multilingual conceptual understanding…
Decomposed Prompting: Probing Multilingual Linguistic Structure Knowledge in Large Language Models
Ercong Nie, Shuzhou Yuan, Bolei Ma +4
Probing the multilingual knowledge of linguistic structure in LLMs, often characterized as sequence labeling, faces challenges with maintaining output templates in current text-to-…