565 citations · 599 across the 13 of their papers we have counts for
6 papers · 1 filter
Bridging the Data Provenance Gap Across Text, Speech and Video
Shayne Longpre, Nikhil Singh, Manuel Cherep +40
Progress in AI is driven largely by the scale and quality of training data. Despite this, there is a deficit of empirical analysis examining the attributes of well-established data…
GitChameleon: Unmasking the Version-Switching Capabilities of Code Generation Models
Nizar Islah, Justine Gehring, Diganta Misra +4
The rapid evolution of software libraries presents a significant challenge for code generation models, which must adapt to frequent version updates while maintaining compatibility…
Consent in Crisis: The Rapid Decline of the AI Data Commons
Shayne Longpre, Robert Mahari, Ariel Lee +46
General-purpose artificial intelligence (AI) systems are built on massive swathes of public web data, assembled into corpora such as C4, RefinedWeb, and Dolma. To our knowledge, we…
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code
Taishi Nakamura, Mayank Mishra, Simone Tedeschi +42
Pretrained language models are an integral part of AI applications, but their high computational cost for training limits accessibility. Initiatives such as Bloom and StarCoder aim…
On the low-shot transferability of [V]-Mamba
Diganta Misra, Jay Gala, Antonio Orvieto
The strength of modern large-scale neural networks lies in their ability to efficiently adapt to new tasks with few examples. Although extensive research has investigated the trans…
Using Shapley interactions to understand how models use structure
Divyansh Singhvi, Diganta Misra, Andrej Erkelens +3
Language is an intricately structured system, and a key goal of NLP interpretability is to provide methodological insights for understanding how language models represent this stru…