5 citations · 8 across the 7 of their papers we have counts for
8 papers
Rao-Blackwellised Reparameterisation Gradients
Kevin H. Lam, Thang D. Bui, George Deligiannidis +1
Latent Gaussian variables have been popularised in probabilistic machine learning. In turn, gradient estimators are the machinery that facilitates gradient-based optimisation for m…
Sparks of Science: Hypothesis Generation Using Structured Paper Data
Charles O'Neill, Tirthankar Ghosal, Roberta Răileanu +4
Generating novel and creative scientific hypotheses is a cornerstone in achieving Artificial General Intelligence. Large language and reasoning models have the potential to aid in…
Sparse Autoencoders Enable Scalable and Reliable Circuit Identification in Language Models
Charles O'Neill, Thang Bui
This paper introduces an efficient and robust method for discovering interpretable circuits in large language models using discrete sparse autoencoders. Our approach addresses key…
Measuring Sharpness in Grokking
Jack Miller, Patrick Gleeson, Charles O'Neill +2
Neural networks sometimes exhibit grokking, a phenomenon where perfect or near-perfect performance is achieved on a validation set well after the same performance has been obtained…
Grokking Beyond Neural Networks: An Empirical Exploration with Model Complexity
Jack Miller, Charles O'Neill, Thang Bui
In some settings neural networks exhibit a phenomenon known as \textit{grokking}, where they achieve perfect or near-perfect accuracy on the validation set long after the same perf…
AstroLLaMA: Towards Specialized Foundation Models in Astronomy
Tuan Dung Nguyen, Yuan-Sen Ting, Ioana Ciucă +21
Large language models excel in many human-language tasks but often falter in highly specialized domains like scholarly astronomy. To bridge this gap, we introduce AstroLLaMA, a 7-b…
Adversarial Fine-Tuning of Language Models: An Iterative Optimisation Approach for the Generation and Detection of Problematic Content
Charles O'Neill, Jack Miller, Ioana Ciuca +2
In this paper, we tackle the emerging challenge of unintended harmful content generation in Large Language Models (LLMs) with a novel dual-stage optimisation technique using advers…
Steering Language Generation: Harnessing Contrastive Expert Guidance and Negative Prompting for Coherent and Diverse Synthetic Data Generation
Charles O'Neill, Yuan-Sen Ting, Ioana Ciuca +2
Large Language Models (LLMs) hold immense potential to generate synthetic data of high quality and utility, which has numerous applications from downstream model training to practi…