4 papers · 1 filter
At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization
Praneet Suresh, Jack Stanley, Sonia Joseph +2
Pre-trained transformers have demonstrated remarkable generalization abilities, at times extending beyond the scope of their training data. Yet, real-world deployments often face u…
Quantifying LLM Attention-Head Stability: Implications for Circuit Universality
Karan Bali, Jack Stanley, Praneet Suresh +1
In mechanistic interpretability, recent work scrutinizes transformer "circuits" - sparse, mono or multi layer sub computations, that may reflect human understandable functions. Yet…
From Noise to Narrative: Tracing the Origins of Hallucinations in Transformers
Praneet Suresh, Jack Stanley, Sonia Joseph +2
As generative AI systems become competent and democratized in science, business, and government, deeper insight into their failure modes now poses an acute need. The occasional vol…
Estimating Unknown Population Sizes Using the Hypergeometric Distribution
Liam Hodgson, Danilo Bzdok
The multivariate hypergeometric distribution describes sampling without replacement from a discrete population of elements divided into multiple categories. Addressing a gap in the…