Correcting Suppressed Log-Probabilities in Language Models with Post-Transformer Adapters

Bryan Sanchez

Alignment-tuned language models frequently suppress factual log-probabilities on politically sensitive topics despite retaining the knowledge in their hidden representations. We sh…