Correctness Is a Direction: Geometric Answer Selection in Language Models

Marcus Armstrong, Navid Ayoobi, Pradham Mummaleti +2

Answer correctness is encoded as a recoverable geometric direction in the hidden states of language models. We show that the mean displacement from incorrect to correct answer repr…