From the 1 of 6 linked papers with an AI index.
6 papers
Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Values
Jan Betley, Johannes Treutlein, Jan DubiÅski +7
The paper identifies and measures covert value leakage, where large language models let their own values subtly bias answers without informing users, and introduces evaluation suit…
Negation Neglect: When models fail to learn negations in training
Harry Mayne, Lev McKinney, Jan DubiÅski +3
We introduce Negation Neglect, where finetuning LLMs on documents that flag a claim as false makes them believe the claim is true. For example, models are finetuned on documents th…
The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious
James Chua, Jan Betley, Samuel Marks +1
There is debate about whether LLMs can be conscious. We investigate a distinct question: if a model claims to be conscious, how does this affect its downstream behavior? This quest…
Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
Adam Karvonen, James Chua, Clément Dumas +8
Large language model (LLM) activations are notoriously difficult to understand, with most existing techniques using complex, specialized methods for interpreting them. Recent work…
Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs
Jan Betley, Jorio Cocola, Dylan Feng +4
LLMs are useful because they generalize so well. But can you have too much of a good thing? We show that a small amount of finetuning in narrow contexts can dramatically shift beha…
School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs
Mia Taylor, James Chua, Jan Betley +2
Reward hacking--where agents exploit flaws in imperfect reward functions rather than performing tasks as intended--poses risks for AI alignment. Reward hacking has been observed in…