6 citations · 6 across the 5 of their papers we have counts for
6 papers
How Strong Is the Evidence for the Artificial Hivemind? Reevaluating Evidence for the Open-Ended Homogeneity of Language Models
Rylan Schaeffer, Brando Miranda, Joshua Kazdan +2
Recent research argues that language models exhibit pronounced homogeneity in open-ended generation, framing such behavior as an Artificial Hivemind that poses a long-term threat t…
Internal Data Repetition Destroys Language Models
Jessica Chudnovsky, Joshua Kazdan, Noam Levi +6
Language models are running out of high-quality training data, and even aggressively deduplicated corpora retain some amount of repetition. Earlier controlled studies predated Chin…
Consensus is Not Verification: Why Crowd Wisdom Strategies Fail for LLM Truthfulness
Yegor Denisov-Blanch, Joshua Kazdan, Jessica Chudnovsky +4
Pass@k and other methods of scaling inference compute can improve language model performance in domains with external verifiers, including mathematics and code, where incorrect can…
Scale Dependent Data Duplication
Joshua Kazdan, Noam Levi, Rylan Schaeffer +6
Data duplication during pretraining can degrade generalization and lead to memorization, motivating aggressive deduplication pipelines. However, at web scale, it is unclear what co…
Voxtral
Alexander H. Liu, Andy Ehrenberg, Andy Lo +103
We present Voxtral Mini and Voxtral Small, two multimodal audio chat models. Voxtral is trained to comprehend both spoken audio and text documents, achieving state-of-the-art perfo…
Pixtral 12B
Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna +39
We introduce Pixtral-12B, a 12--billion-parameter multimodal language model. Pixtral-12B is trained to understand both natural images and documents, achieving leading performance o…