Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs

Ankush Checkervarty

Activation probes that monitor deployed language models are trained on synthetic conversations, and how many a probe needs is open. We trace learning curves over 10-590 synthetic s…