Alignment Forecasting: Predicting Misalignment From Training Data

Chen Yueh-Han, Bruce W. Lee, Ilia Sucholutsky +1

Training a language model on data with a narrow flaw can sometimes make the model broadly misaligned. Inspecting the data at face value often does not settle whether it will emerge…