563 citations
- FACE FoundationUS8 papers
- Stanford UniversityUS5 papers
- Cornell UniversityUS4 papers
- Massachusetts Institute of TechnologyUS4 papers
- Allen Institute for Artificial IntelligenceUS3 papers
- Carnegie Mellon UniversityUS3 papers
- Google (United States)US3 papers
- King Fahd University of Petroleum and MineralsSA3 papers
- Sorbonne UniversitéFR3 papers
- Allen InstituteUS2 papers
- Booz Allen Hamilton (United States)US2 papers
- École PolytechniqueFR2 papers
6 papers · 1 filter
GemMaroc: Unlocking Darija Proficiency in LLMs with Minimal Data
Abderrahman Skiredj, Ferdaous Azhari, Houdaifa Atou +2
Open-source large language models (LLMs) still marginalise Moroccan Arabic (Darija), forcing practitioners either to bolt on heavyweight Arabic adapters or to sacrifice the very re…
Energy and Carbon Considerations of Fine-Tuning BERT
Xiaorong Wang, Clara Na, Emma Strubell +2
Despite the popularity of the `pre-train then fine-tune' paradigm in the NLP community, existing work quantifying energy costs and associated carbon emissions has largely focused o…
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Hugo Laurençon, Lucile Saulnier, Thomas Wang +51
As language models grow ever larger, the need for large-scale high-quality text datasets has never been more pressing, especially in multilingual settings. The BigScience workshop,…
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
Sabrina J. Mielke, Zaid Alyafeai, Elizabeth Salesky +8
What are the units of text that we want to model? From bytes to multi-word expressions, text can be analyzed and generated at many granularities. Until recently, most natural langu…
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Julia Kreutzer, Isaac Caswell, Lisa Wang +49
With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text dataset…
Learning from others' mistakes: Avoiding dataset biases without modeling them
Victor Sanh, Thomas Wolf, Yonatan Belinkov +1
State-of-the-art natural language processing (NLP) models often learn to model dataset biases and surface form correlations instead of features that target the intended underlying…