How High Is 0.6? Floors, Ceilings, and Headroom in Interpretability Probing

Pranjal Garg

Probes are the workhorse of interpretability. If a model's hidden states predict a variable, the model is said to represent it. But a probe score has no fixed meaning. An $R^2$ of…