Feeding LLM Annotations to BERT Classifiers at Your Own Risk

Yucheng Lu, Kazimier Smith

Using LLM-generated labels to fine-tune smaller encoder-only models for text classification has gained popularity in various settings. While this approach may be justified in simpl…