Cross-Model In-Context Neurofeedback Reveals (Limited) Activation Control in LLMs

Davide Baldelli, Robyn Kozierok, Mirko Bronzi +1

Probes on activation vectors are widely used to detect misbehavior in language models, but their success relies on the assumption that models cannot control what their activations…