133 citations · 281 across the 44 of their papers we have counts for
26 papers · 1 filter
How Strong Is the Evidence for the Artificial Hivemind? Reevaluating Evidence for the Open-Ended Homogeneity of Language Models
Rylan Schaeffer, Brando Miranda, Joshua Kazdan +2
Recent research argues that language models exhibit pronounced homogeneity in open-ended generation, framing such behavior as an Artificial Hivemind that poses a long-term threat t…
Internal Data Repetition Destroys Language Models
Jessica Chudnovsky, Joshua Kazdan, Noam Levi +6
Language models are running out of high-quality training data, and even aggressively deduplicated corpora retain some amount of repetition. Earlier controlled studies predated Chin…
Item Response Scaling Laws: A Measurement Theory Approach for Efficient and Generalizable Neural Scaling Estimation
Sang Truong, Yuheng Tu, Rylan Schaeffer +1
Scaling laws provide a fundamental framework for understanding the performance of Language Models (LMs), yet deriving them requires prohibitively expensive evaluations across thous…
Consensus is Not Verification: Why Crowd Wisdom Strategies Fail for LLM Truthfulness
Yegor Denisov-Blanch, Joshua Kazdan, Jessica Chudnovsky +4
Pass@k and other methods of scaling inference compute can improve language model performance in domains with external verifiers, including mathematics and code, where incorrect can…
Scale Dependent Data Duplication
Joshua Kazdan, Noam Levi, Rylan Schaeffer +6
Data duplication during pretraining can degrade generalization and lead to memorization, motivating aggressive deduplication pipelines. However, at web scale, it is unclear what co…
Quantifying the Effect of Test Set Contamination on Generative Evaluations
Rylan Schaeffer, Joshua Kazdan, Baber Abbasi +8
As frontier AI systems are pretrained on web-scale data, test set contamination has become a critical concern for accurately assessing their capabilities. While research has thorou…
Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed
Isha Gupta, Rylan Schaeffer, Joshua Kazdan +2
The field of adversarial robustness has long established that adversarial examples can successfully transfer between image classifiers and that text jailbreaks can successfully tra…
Evaluating the Robustness of Chinchilla Compute-Optimal Scaling
Rylan Schaeffer, Noam Levi, Andreas Kirsch +4
Hoffman et al (2022)'s Chinchilla paper introduced the principle of compute-optimal scaling, laying a foundation for future scaling of language models. In the years since, however,…
Pretraining Scaling Laws for Generative Evaluations of Language Models
Rylan Schaeffer, Noam Levi, Brando Miranda +1
Neural scaling laws have driven the field's ever-expanding exponential growth in parameters, data and compute. While scaling behaviors for pretraining losses and discriminative ben…
Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
Rylan Schaeffer, Joshua Kazdan, Yegor Denisov-Blanch +11
Science progresses by iteratively advancing and correcting humanity's understanding of the world. In machine learning (ML) research, rapid advancements have led to an explosion of…
Position: Model Collapse Does Not Mean What You Think
Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu +1
The proliferation of AI-generated content online has fueled concerns over \emph{model collapse}, a degradation in future generative models' performance when trained on synthetic da…
Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach
Tony T. Wang, John Hughes, Henry Sleight +7
Defending large language models against jailbreaks so that they never engage in a broadly-defined set of forbidden behaviors is an open problem. In this paper, we investigate the d…
ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment
Elyas Obbad, Iddah Mlauzi, Brando Miranda +4
Data selection is crucial for optimizing language model (LM) performance on specific tasks, yet most existing methods fail to effectively consider the target task distribution. Cur…
Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
Joshua Kazdan, Rylan Schaeffer, Apratim Dey +4
What happens when generative machine learning models are pretrained on web-scale datasets containing data generated by earlier models? Some prior work warns of "model collapse" as…
In-Context Learning of Energy Functions
Rylan Schaeffer, Mikail Khona, Sanmi Koyejo
In-context learning is a powerful capability of certain machine learning models that arguably underpins the success of today's frontier AI models. However, in-context learning is c…
Quantifying Variance in Evaluation Benchmarks
Lovish Madaan, Aaditya K. Singh, Rylan Schaeffer +5
Evaluation benchmarks are the cornerstone of measuring capabilities of large language models (LLMs), as well as driving progress in said capabilities. Originally designed to make c…
Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations
Rylan Schaeffer, Victor Lecomte, Dhruv Bhandarkar Pai +10
Maximum Manifold Capacity Representations (MMCR) is a recent multi-view self-supervised learning (MVSSL) method that matches or surpasses other leading MVSSL methods. MMCR is intri…
Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
Rylan Schaeffer, Hailey Schoelkopf, Brando Miranda +6
Predicting changes from scaling advanced AI systems is a desirable property for engineers, economists, governments and industry alike, and, while a well-established literature exis…
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey +11
The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated out…
Bridging Associative Memory and Probabilistic Modeling
Rylan Schaeffer, Nika Zahedi, Mikail Khona +9
Associative memory and probabilistic modeling are two fundamental topics in artificial intelligence. The first studies recurrent neural networks designed to denoise, complete and r…
What Causes Polysemanticity? An Alternative Origin Story of Mixed Selectivity from Incidental Causes
Victor Lecomte, Kushal Thaman, Rylan Schaeffer +3
Polysemantic neurons -- neurons that activate for a set of unrelated features -- have been seen as a significant obstacle towards interpretability of task-optimized deep networks,…
Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells
Rylan Schaeffer, Mikail Khona, Tzuhsuan Ma +3
To solve the spatial problems of mapping, localization and navigation, the mammalian lineage has developed striking spatial representations. One important spatial representation is…
Deceptive Alignment Monitoring
Andres Carranza, Dhruv Pai, Rylan Schaeffer +2
As the capabilities of large machine learning models continue to grow, and as the autonomy afforded to such models continues to expand, the spectre of a new adversary looms: the mo…
FACADE: A Framework for Adversarial Circuit Anomaly Detection and Evaluation
Dhruv Pai, Andres Carranza, Rylan Schaeffer +2
We present FACADE, a novel probabilistic and geometric framework designed for unsupervised mechanistic anomaly detection in deep neural networks. Its primary goal is advancing the…