190 citations · 215 across the 11 of their papers we have counts for
11 papers
Consistent Explanations in the Face of Model Indeterminacy via Ensembling
Dan Ley, Leonard Tang, Matthew Nazari +3
This work addresses the challenge of providing consistent explanations for predictive models in the presence of model indeterminacy, which arises due to the existence of multiple (…
Degraded Polygons Raise Fundamental Questions of Neural Network Perception
Leonard Tang, Dan Ley
It is well-known that modern computer vision systems often exhibit behaviors misaligned with those of humans: from adversarial attacks to image corruptions, deep learning vision mo…
Baselines for Identifying Watermarked Large Language Models
Leonard Tang, Gavin Uberti, Tom Shlomi
We consider the emerging problem of identifying the presence and use of watermarking schemes in widely used, publicly hosted, closed source large language models (LLMs). We introdu…
Learning the Wrong Lessons: Inserting Trojans During Knowledge Distillation
Leonard Tang, Tom Shlomi, Alexander Cai
In recent years, knowledge distillation has become a cornerstone of efficiently deployed machine learning, with labs and industries using knowledge distillation to train models tha…
MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding
Steven H. Wang, Antoine Scardigli, Leonard Tang +7
Reading comprehension of legal text can be a particularly challenging task due to the length and complexity of legal clauses and a shortage of expert-annotated datasets. To address…
The Naughtyformer: A Transformer Understands Offensive Humor
Leonard Tang, Alexander Cai, Steve Li +1
Jokes are intentionally written to be funny, but not all jokes are created the same. Some jokes may be fit for a classroom of kindergarteners, but others are best reserved for a mo…
Lila: A Unified Benchmark for Mathematical Reasoning
Swaroop Mishra, Matthew Finlayson, Pan Lu +8
Mathematical reasoning skills are essential for general-purpose intelligent systems to perform tasks from grocery shopping to climate modeling. Towards evaluating and improving AI…
From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams
Iddo Drori, Sarah J. Zhang, Reece Shuttleworth +13
A final exam in machine learning at a top institution such as MIT, Harvard, or Cornell typically takes faculty days to write, and students hours to solve. We demonstrate that large…
A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level
Iddo Drori, Sarah Zhang, Reece Shuttleworth +15
We demonstrate that a neural network pre-trained on text and fine-tuned on code solves mathematics course problems, explains solutions, and generates new questions at a human level…
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
Dan Hendrycks, Andy Zou, Mantas Mazeika +4
In real-world applications of machine learning, reliable and safe systems must consider measures of performance beyond standard test set accuracy. These other goals include out-of-…
Solving Probability and Statistics Problems by Program Synthesis
Leonard Tang, Elizabeth Ke, Nikhil Singh +2
We solve university level probability and statistics questions by program synthesis using OpenAI's Codex, a Transformer trained on text and fine-tuned on code. We transform course…