Institutional Books 1.0: A 242B token dataset from Harvard Library's collections, refined for accuracy and usability

Matteo Cargnelutti, Catherine Brobston, John Hess +8

Large language models (LLMs) use data to learn about the world in order to produce meaningful correlations and predictions. As such, the nature, scale, quality, and diversity of th…