1 citations · 1 across the 3 of their papers we have counts for
4 papers
Mitigating Reward Hacking in Realistic Scenarios Through Representation Engineering
Austin Meek, Kyle Cox, Matthew Nguyen +2
Reward hacking is a major problem in current production machine learning systems, resulting in costly and dangerous failures. Here we study how to monitor for and mitigate reward h…
On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness
Matthew Nguyen, Kyle Cox, Austin Meek +1
Model capabilities have improved in large part due to scaling chain of thought. This has been a promising development for AI safety--where models verbalize their reasoning, it is p…
Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers
Kyle Cox, Darius Kianersi, Adrià Garriga-Alonso
As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, sugges…
Mapping from Meaning: Addressing the Miscalibration of Prompt-Sensitive Language Models
Kyle Cox, Jiawei Xu, Yikun Han +6
An interesting behavior in large language models (LLMs) is prompt sensitivity. When provided with different but semantically equivalent versions of the same prompt, models may prod…