3 citations · 3 across the 5 of their papers we have counts for
5 papers
Split Personality Training: Revealing Latent Knowledge Through Alternate Personalities
Florian Dietz, William Wale, Oscar Gilg +5
Detecting misalignment in large language models is challenging because models may learn to conceal misbehavior during training. Standard auditing techniques fall short: black-box m…
Auditing language models for hidden objectives
Samuel Marks, Johannes Treutlein, Trenton Bricken +32
We study the feasibility of conducting alignment audits: investigations into whether models have undesired objectives. As a testbed, we train a language model with a hidden objecti…
IGC: Integrating a Gated Calculator into an LLM to Solve Arithmetic Tasks Reliably and Efficiently
Florian Dietz, Dietrich Klakow
Solving arithmetic tasks is a simple and fundamental skill, yet modern Large Language Models (LLMs) have great difficulty with them. We introduce the Integrated Gated Calculator (I…
Block-Operations: Using Modular Routing to Improve Compositional Generalization
Florian Dietz, Dietrich Klakow
We explore the hypothesis that poor compositional generalization in neural networks is caused by difficulties with learning effective routing. To solve this problem, we propose the…
Comgra: A Tool for Analyzing and Debugging Neural Networks
Florian Dietz, Sophie Fellenz, Dietrich Klakow +1
Neural Networks are notoriously difficult to inspect. We introduce comgra, an open source python library for use with PyTorch. Comgra extracts data about the internal activations o…