6 citations · 6 across the 6 of their papers we have counts for
7 papers
What Accuracy Hides In Learned Transformer Programs
Jonathan Lee, Narmeen Oozeer, Philip Quirke
Transformer Programs (TP) (Friedman et al., 2023) constrain a Transformer's architecture so that its learned computation can be extracted as readable Python code. TP expose learned…
Necessary, Decodable and Reversible, Yet Not Transferable: A Stress Test for Attention-Head Role Claims
Philip Quirke
Mechanistic studies often assign a component a role when removing it damages a behavior, its activation linearly encodes task information, and restoring that activation repairs the…
Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing
Michael Lan, Narmeen Fatimah Oozeer, Chaithanya Bandi +4
While mechanistic interpretability (MI) has produced important insights into neural network internals, the field has yet to establish a standardized system to audit experiments. As…
Planning in a recurrent neural network that plays Sokoban
Mohammad Taufeeque, Philip Quirke, Maximilian Li +4
Planning is essential for solving complex tasks, yet the internal mechanisms underlying planning in neural networks remain poorly understood. Building on prior work, we analyze a r…
Understanding Addition and Subtraction in Transformers
Philip Quirke, Clement Neo, Fazl Barez
We use integer addition and subtraction as a controlled, exactly-solvable testbed for what can be said with confidence about the algorithm a low-loss transformer implements - logic…
Understanding Addition in Transformers
Philip Quirke, Fazl Barez
Understanding the inner workings of machine learning models like Transformers is vital for their safe and ethical use. This paper provides a comprehensive analysis of a one-layer T…
Fully transformer-based biomarker prediction from colorectal cancer histology: a large-scale multicentric study
Sophia J. Wagner, Daniel Reisenbüchler, Nicholas P. West +28
Background: Deep learning (DL) can extract predictive and prognostic biomarkers from routine pathology slides in colorectal cancer. For example, a DL test for the diagnosis of micr…