8 papers · 1 filter
MURANO: Design, Run, and Reproduce Mechanistic Interpretability Experiments as Composable Pipelines
Alireza Bayat Makou, Emirhan Böge, Phu Gia Hoang +5
This paper presents Murano, an open source framework for designing, running, and reproducing mechanistic interpretability studies of large language models, intended for researchers…
Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery
Alireza Bayat Makou, Jingcheng Niu, Subhabrata Dutta +1
Circuit discovery methods identify subgraphs that explain model behaviors, and structural differences between discovered circuits are commonly interpreted as evidence of distinct m…
Hierarchical Latent Structures in Data Generation Process Unify Mechanistic Phenomena across Scale
Jonas Rohweder, Subhabrata Dutta, Iryna Gurevych
Contemporary studies in mechanistic interpretability have uncovered many puzzling phenomena in the neural information processing of Transformer-based language models, such as induc…
Patches of Nonlinearity: Instruction Vectors in Large Language Models
Irina Bigoulaeva, Jonas Rohweder, Subhabrata Dutta +1
Despite the recent success of instruction-tuned language models and their ubiquitous usage, very little is known of how models process instructions internally. In this work, we add…
Reward Modeling for Scientific Writing Evaluation
Furkan Şahinuç, Subhabrata Dutta, Iryna Gurevych
Scientific writing is an expert-domain task that demands deep domain knowledge, task-specific requirements and reasoning capabilities that leverage the domain knowledge to satisfy…
Expert Preference-based Evaluation of Automated Related Work Generation
Furkan Şahinuç, Subhabrata Dutta, Iryna Gurevych
Expert domain writing, such as scientific writing, typically demands extensive domain knowledge. Although large language models (LLMs) show promising potential in this task, evalua…
Factual Self-Awareness in Language Models: Representation, Robustness, and Scaling
Hovhannes Tamoyan, Subhabrata Dutta, Iryna Gurevych
Factual incorrectness in generated content is one of the primary concerns in ubiquitous deployment of large language models (LLMs). Prior findings suggest LLMs can (sometimes) dete…
Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning
Jingcheng Niu, Subhabrata Dutta, Ahmed Elshabrawy +2
Large-scale Transformer language models (LMs) trained solely on next-token prediction with web-scale data can solve a wide range of tasks after seeing just a few examples. The mech…