Rephrasing natural text data with different languages and quality levels for Large Language Model pre-training

Michael Pieler, Marco Bellagente, Hannah Teufel +9

Recently published work on rephrasing natural text data for pre-training LLMs has shown promising results when combining the original dataset with the synthetically rephrased data.…