4 papers
iPOE: Interpretable Prompt Optimization via Explanations
Jiahui Li, Yarik Menchaca Resendiz, Sean Papay +1
Prompt optimization has often been framed as a discrete search problem to find high-performing and robust instructions for an LLM. However, the search result might not make it tran…
Are Humans as Brittle as Large Language Models?
Jiahui Li, Sean Papay, Roman Klinger
The output of large language models (LLMs) is unstable, due both to non-determinism of the decoding process as well as to prompt brittleness. While the intrinsic non-determinism of…
iPrOp: Interactive Prompt Optimization for Large Language Models with a Human in the Loop
Jiahui Li, Roman Klinger
Prompt engineering has made significant contributions to the era of large language models, yet its effectiveness depends on the skills of a prompt author. This paper introduces $\t…
Which Demographics do LLMs Default to During Annotation?
Johannes Schäfer, Aidan Combs, Christopher Bagdon +9
Demographics and cultural background of annotators influence the labels they assign in text annotation -- for instance, an elderly woman might find it offensive to read a message a…