72 citations · 108 across the 10 of their papers we have counts for
4 papers · 1 filter
Forecasting Frontier Language Model Agent Capabilities
Govind Pimpale, Axel Højmark, Jérémy Scheurer +1
As Language Models (LMs) increasingly operate as autonomous agents, accurately forecasting their capabilities becomes crucial for societal preparedness. We evaluate six forecasting…
TracrBench: Generating Interpretability Testbeds with Large Language Models
Hannes Thurnherr, Jérémy Scheurer
Achieving a mechanistic understanding of transformer-based language models is an open challenge, especially due to their large number of parameters. Moreover, the lack of ground tr…
Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs
Rudolf Laine, Bilal Chughtai, Jan Betley +6
AI assistants such as ChatGPT are trained to respond to users by saying, "I am a large language model". This raises questions. Do such models know that they are LLMs and reliably a…
Large Language Models can Strategically Deceive their Users when Put Under Pressure
Jérémy Scheurer, Mikita Balesni, Marius Hobbhahn
We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about…