Toxicity of the Commons: Curating Open-Source Pre-Training Data

Catherine Arnett, Eliot Jones, Ivan P. Yamshchikov +1

Open-source large language models are becoming increasingly available and popular among researchers and practitioners. While significant progress has been made on open-weight model…