Inoculation Midtraining with Learned Neologisms

Kyle O'Brien, Edward James Young, Puria Radmard +4

Large language models (LLMs) often learn both desirable and undesirable properties during post-training. We study whether midtraining, an earlier training stage, can shape which of…