Listening to the Wise Few: Select-and-Copy Attention Heads for Multiple-Choice QA

Eduard Tulchinskii, Laida Kushnareva, Kristian Kuznetsov +5

A standard way to evaluate the abilities of LLM involves presenting a multiple-choice question and selecting the option with the highest logit as the model's predicted answer. Howe…