On-Policy Consistency Training Improves LLM Safety with Minimal Capability Degradation

Andy Han, Kristina Fujimoto, Avidan Shah +5

Aligned models can misbehave in several ways: they are often sycophantic, fall victim to jailbreaks, or fail to include appropriate safety warnings. Consistency training is a promi…