23 citations · 30 across the 7 of their papers we have counts for
7 papers
Automated alignment is harder than you think
Aleksandr Bowkis, Marie Davidsen Buhl, Jacob Pfau +1
A leading proposal for aligning artificial superintelligence (ASI) is to use AI agents to automate an increasing fraction of alignment research as capabilities improve. We argue th…
An alignment safety case sketch based on debate
Marie Davidsen Buhl, Jacob Pfau, Benjamin Hilton +1
If AI systems match or exceed human capabilities on a wide range of tasks, it may become difficult for humans to efficiently judge their actions -- making it hard to use human feed…
When Benchmarks Talk: Re-Evaluating Code LLMs with Interactive Feedback
Jane Pan, Ryan Shar, Jacob Pfau +3
Programming is a fundamentally interactive process, yet coding assistants are often evaluated using static benchmarks that fail to measure how well models collaborate with users. W…
Taking AI Welfare Seriously
Robert Long, Jeff Sebo, Patrick Butlin +7
In this report, we argue that there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future. That means that the prospect of AI…
Steering Without Side Effects: Improving Post-Deployment Control of Language Models
Asa Cooper Stickland, Alexander Lyzhov, Jacob Pfau +2
Language models (LMs) have been shown to behave unexpectedly post-deployment. For example, new jailbreaks continually arise, allowing model misuse, despite extensive red-teaming an…
Let's Think Dot by Dot: Hidden Computation in Transformer Language Models
Jacob Pfau, William Merrill, Samuel R. Bowman
Chain-of-thought responses from language models improve performance across most benchmarks. However, it remains unclear to what extent these performance gains can be attributed to…
Self-Consistency of Large Language Models under Ambiguity
Henning Bartsch, Ole Jorgensen, Domenic Rosati +2
Large language models (LLMs) that do not give consistent answers across contexts are problematic when used for tasks with expectations of consistency, e.g., question-answering, exp…