18 citations · 31 across the 25 of their papers we have counts for
11 papers · 1 filter
Reliable but Design-Sensitive: Instrument Uncertainty in LLM Annotation
Thomas Reiter, Christoph Kern, Fedor Miasnikov +4
Large language models (LLMs) can give reliable labels under one setup yet change those labels when researchers make other reasonable design choices. We tested seven LLMs, 12 task d…
Moral Lenses, Political Coordinates: Towards Ideological Positioning of Morally Conditioned LLMs
Chenchen Yuan, Bolei Ma, Zheyu Zhang +3
While recent research has systematically documented political orientation in large language models (LLMs), existing evaluations rely primarily on direct probing or demographic pers…
Toward Understanding the Transferability of Adversarial Suffixes in Large Language Models
Sarah Ball, Niki Hasrati, Alexander Robey +4
Discrete optimization-based jailbreaking attacks on large language models aim to generate short, nonsensical suffixes that, when appended onto input prompts, elicit disallowed cont…
Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation
Bolei Ma, Yong Cao, Indira Sen +4
Large Language Models (LLMs) are increasingly used to simulate public opinion and other social phenomena. Most current studies constrain these simulations to multiple-choice or sho…
Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges
Bolei Ma, Yuting Li, Wei Zhou +7
Understanding pragmatics-the use of language in context-is crucial for developing NLP systems capable of interpreting nuanced language use. Despite recent advances in language tech…
Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study
Bolei Ma, Berk Yoztyurk, Anna-Carolina Haensch +5
In recent research, large language models (LLMs) have been increasingly used to investigate public opinions. This study investigates the algorithmic fidelity of LLMs, i.e., the abi…
The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models
Bolei Ma, Xinpeng Wang, Tiancheng Hu +4
Recent advances in Large Language Models (LLMs) have sparked wide interest in validating and comprehending the human-like cognitive-behavioral traits LLMs may capture and convey. T…
Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models
Sarah Ball, Frauke Kreuter, Nina Panickssery
Conversational large language models are trained to refuse to answer harmful questions. However, emergent jailbreaking techniques can still elicit unsafe outputs, presenting an ong…
Decomposed Prompting: Probing Multilingual Linguistic Structure Knowledge in Large Language Models
Ercong Nie, Shuzhou Yuan, Bolei Ma +4
Probing the multilingual knowledge of linguistic structure in LLMs, often characterized as sequence labeling, faces challenges with maintaining output templates in current text-to-…
"My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models
Xinpeng Wang, Bolei Ma, Chengzhi Hu +5
The open-ended nature of language generation makes the evaluation of autoregressive large language models (LLMs) challenging. One common evaluation approach uses multiple-choice qu…
ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks
Bolei Ma, Ercong Nie, Shuzhou Yuan +4
Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding. However, most previous studies primarily f…