1 citations · 1 across the 10 of their papers we have counts for
11 papers
Diff Mining: Logit Differences Reveal Finetuning Objectives
Greg Kocher, Robert West, Clément Dumas +1
Finetuning has become the gold standard for refining existing behaviors and inducing new ones in language models, yet it often remains unclear exactly which behaviors emerge during…
Synthetic Persona Pretraining: Alignment from Token Zero
Julian Minder, Viktor Moskvoretskii, Raghav Singhal +12
As language-model-based AI is increasingly deployed in autonomous settings, aligning its goals and values with those of humans becomes critical. Today, alignment, and the assistant…
Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
Adam Karvonen, James Chua, Clément Dumas +8
Large language model (LLM) activations are notoriously difficult to understand, with most existing techniques using complex, specialized methods for interpreting them. Recent work…
Believe It or Not: How Deeply do LLMs Believe Implanted Facts?
Stewart Slocum, Julian Minder, Clément Dumas +4
Knowledge editing techniques promise to implant new factual knowledge into large language models (LLMs). But do LLMs really believe these facts? We develop a framework to measure b…
Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences
Julian Minder, Clément Dumas, Stewart Slocum +4
Finetuning on narrow domains has become an essential tool to adapt Large Language Models (LLMs) to specific tasks and to create models with known unusual properties that are useful…
The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?
Denis Sutter, Julian Minder, Thomas Hofmann +1
The concept of causal abstraction got recently popularised to demystify the opaque decision-making processes of machine learning models; in short, a neural network can be abstracte…
Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning
Julian Minder, Clément Dumas, Caden Juang +2
Model diffing is the study of how fine-tuning changes a model's representations and internal algorithms. Many behaviors of interest are introduced during fine-tuning, and model dif…
Controllable Context Sensitivity and the Knob Behind It
Julian Minder, Kevin Du, Niklas Stoehr +4
When making predictions, a language model must trade off how much it relies on its context vs. its prior knowledge. Choosing how sensitive the model is to its context is a fundamen…
Data2Neo - A Tool for Complex Neo4j Data Integration
Julian Minder, Laurence Brandenberger, Luis Salamanca +1
This paper introduces Data2Neo, an open-source Python library for converting relational data into knowledge graphs stored in Neo4j databases. With extensive customization options a…
SALSA-CLRS: A Sparse and Scalable Benchmark for Algorithmic Reasoning
Julian Minder, Florian Grötschla, Joël Mathys +1
We introduce an extension to the CLRS algorithmic learning benchmark, prioritizing scalability and the utilization of sparse representations. Many algorithms in CLRS require global…
Fast Record Linkage for Company Entities
Thomas Gschwind, Christoph Miksovic, Julian Minder +2
Record linkage is an essential part of nearly all real-world systems that consume structured and unstructured data coming from different sources. Typically no common key is availab…