Failure of contextual invariance in large language models

Sagar Kumar, Ariel Flint, Luca Maria Aiello +1

Standard evaluation practices assume that large language model (LLM) outputs are stable when prompts are embedded in contextually equivalent discourses. Here, we test this assumpti…