11 papers
Image, Word and Thought: A More Challenging Language Task for the Iterated Learning Model
Hyoyeon Lee, Seth Bullock, Conor Houghton
The iterated learning model simulates the transmission of language from generation to generation in order to explore how the constraints imposed by language transmission facilitate…
Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations
Lucy Farnik, Tim Lawson, Conor Houghton +1
Sparse autoencoders (SAEs) have been successfully used to discover sparse and human-interpretable representations of the latent activations of LLMs. However, we would ultimately li…
Modeling Nonlinear Oscillator Networks Using Physics-Informed Hybrid Reservoir Computing
Andrew Shannon, Conor Houghton, David Barton +1
Surrogate modeling of non-linear oscillator networks remains challenging due to discrepancies between simplified analytical models and real-world complexity. To bridge this gap, we…
Cooperation guides evolution in a minimal model of biological evolution
Conor Houghton
A challenging simulation of evolutionary dynamics based on a three-state cellular automaton is used as a test of how cooperation can drive the evolution of complex traits. Building…
Residual Stream Analysis with Multi-Layer SAEs
Tim Lawson, Lucy Farnik, Conor Houghton +1
Sparse autoencoders (SAEs) are a promising approach to interpreting the internal representations of transformer language models. However, SAEs are usually trained separately on eac…
An iterated learning model of language change that mixes supervised and unsupervised learning
Jack Bunyan, Seth Bullock, Conor Houghton
The iterated learning model is an agent model which simulates the transmission of of language from generation to generation. It is used to study how the language adapts to pressure…