4 citations · 11 across the 6 of their papers we have counts for
6 papers
Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities
Andrey Anurin, Jonathan Ng, Kibo Schaffer +2
LLM agents have the potential to revolutionize defensive cyber operations, but their offensive capabilities are not yet fully understood. To prepare for emerging threats, model dev…
Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts
Jacob Haimes, Cenny Wenner, Kunvar Thaman +4
The training data for many Large Language Models (LLMs) is contaminated with test data. This means that public benchmarks used to assess LLMs are compromised, suggesting a performa…
DeepDecipher: Accessing and Investigating Neuron Activation in Large Language Models
Albert Garde, Esben Kran, Fazl Barez
As large language models (LLMs) become more capable, there is an urgent need for interpretable and transparent tools. Current methods are difficult to implement, and accessible too…
Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark
Jason Hoelscher-Obermaier, Julia Persson, Esben Kran +2
Recent model editing techniques promise to mitigate the problem of memorizing false or outdated associations during LLM training. However, we show that these techniques can introdu…
Neuron to Graph: Interpreting Language Model Neurons at Scale
Alex Foote, Neel Nanda, Esben Kran +3
Advances in Large Language Models (LLMs) have led to remarkable capabilities, yet their inner mechanisms remain largely unknown. To understand these models, we need to unravel the…
N2G: A Scalable Approach for Quantifying Interpretable Neuron Representations in Large Language Models
Alex Foote, Neel Nanda, Esben Kran +2
Understanding the function of individual neurons within language models is essential for mechanistic interpretability research. We propose , a tool…