Consistency Training Along the Transformer Stack

Sukrati Gautam, Neil Shah, Arav Dhoot +7

Consistency training encourages models to behave similarly across different contexts, and has shown promise for reducing misalignment. We broaden the scope of consistency training…