1 citations · 1 across the 2 of their papers we have counts for
6 papers
Internal Data Repetition Destroys Language Models
Jessica Chudnovsky, Joshua Kazdan, Noam Levi +6
Language models are running out of high-quality training data, and even aggressively deduplicated corpora retain some amount of repetition. Earlier controlled studies predated Chin…
Consensus is Not Verification: Why Crowd Wisdom Strategies Fail for LLM Truthfulness
Yegor Denisov-Blanch, Joshua Kazdan, Jessica Chudnovsky +4
Pass@k and other methods of scaling inference compute can improve language model performance in domains with external verifiers, including mathematics and code, where incorrect can…
Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
Rylan Schaeffer, Joshua Kazdan, Yegor Denisov-Blanch +11
Science progresses by iteratively advancing and correcting humanity's understanding of the world. In machine learning (ML) research, rapid advancements have led to an explosion of…
Min-p, Max Exaggeration: A Critical Analysis of Min-p Sampling in Language Models
Rylan Schaeffer, Joshua Kazdan, Yegor Denisov-Blanch
Sampling from language models impacts the quality and diversity of outputs, affecting both research and real-world applications. Recently, Nguyen et al. 2024's "Turning Up the Heat…
Measuring Determinism in Large Language Models for Software Code Review
Eugene Klishevich, Yegor Denisov-Blanch, Simon Obstbaum +2
Large Language Models (LLMs) promise to streamline software code reviews, but their ability to produce consistent assessments remains an open question. In this study, we tested fou…
Predicting Expert Evaluations in Software Code Reviews
Yegor Denisov-Blanch, Igor Ciobanu, Simon Obstbaum +1
Manual code reviews are an essential but time-consuming part of software development, often leading reviewers to prioritize technical issues while skipping valuable assessments. Th…