Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling

Jannek Ulm, Kevin Du, Vésteinn Snæbjarnarson

Large language models (LLMs) are trained on huge amounts of textual data, and concerns have been raised that the limits of such data may soon be reached. A potential solution is to…