190 citations · 215 across the 11 of their papers we have counts for
7 papers · 1 filter
Consistent Explanations in the Face of Model Indeterminacy via Ensembling
Dan Ley, Leonard Tang, Matthew Nazari +3
This work addresses the challenge of providing consistent explanations for predictive models in the presence of model indeterminacy, which arises due to the existence of multiple (…
Baselines for Identifying Watermarked Large Language Models
Leonard Tang, Gavin Uberti, Tom Shlomi
We consider the emerging problem of identifying the presence and use of watermarking schemes in widely used, publicly hosted, closed source large language models (LLMs). We introdu…
Learning the Wrong Lessons: Inserting Trojans During Knowledge Distillation
Leonard Tang, Tom Shlomi, Alexander Cai
In recent years, knowledge distillation has become a cornerstone of efficiently deployed machine learning, with labs and industries using knowledge distillation to train models tha…
From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams
Iddo Drori, Sarah J. Zhang, Reece Shuttleworth +13
A final exam in machine learning at a top institution such as MIT, Harvard, or Cornell typically takes faculty days to write, and students hours to solve. We demonstrate that large…
A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level
Iddo Drori, Sarah Zhang, Reece Shuttleworth +15
We demonstrate that a neural network pre-trained on text and fine-tuned on code solves mathematics course problems, explains solutions, and generates new questions at a human level…
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
Dan Hendrycks, Andy Zou, Mantas Mazeika +4
In real-world applications of machine learning, reliable and safe systems must consider measures of performance beyond standard test set accuracy. These other goals include out-of-…
Solving Probability and Statistics Problems by Program Synthesis
Leonard Tang, Elizabeth Ke, Nikhil Singh +2
We solve university level probability and statistics questions by program synthesis using OpenAI's Codex, a Transformer trained on text and fine-tuned on code. We transform course…