6 papers
Inoculation Midtraining with Learned Neologisms
Kyle O'Brien, Edward James Young, Puria Radmard +4
Large language models (LLMs) often learn both desirable and undesirable properties during post-training. We study whether midtraining, an earlier training stage, can shape which of…
Understanding Goal Generalisation in Sequential Reinforcement Learning
Jason Ross Brown, Edward James Young
Reinforcement learning agents often exhibit unintended goal-directed behaviour outside their training distribution, but we currently lack a principled understanding of how such age…
A transformer architecture alteration to incentivise externalised reasoning
Elizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan +3
We propose a new architectural change, and post-training pipeline, for making LLMs more verbose reasoners by teaching a model to truncate forward passes early. We augment an existi…
Questionnaire Responses Do not Capture the Safety of AI Agents
Max Hellrigel-Holderbaum, Edward James Young
As AI systems advance in capabilities, measuring their safety and alignment to human values is becoming paramount. A fast-growing field of AI research is devoted to developing such…
Diagnosing Pathological Chain-of-Thought in Reasoning Models
Manqing Liu, David Williams-King, Ida Caspary +5
Chain-of-thought (CoT) reasoning is fundamental to modern LLM architectures and represents a critical intervention point for AI safety. However, CoT reasoning may exhibit failure m…
Large language models can learn and generalize steganographic chain-of-thought under process supervision
Joey Skaf, Luis Ibanez-Lissen, Robert McCarthy +8
Chain-of-thought (CoT) reasoning not only enhances large language model performance but also provides critical insights into decision-making processes, marking it as a useful tool…