157 citations · 402 across the 21 of their papers we have counts for
21 papers
The Approximation Ratio for the Risk of Myopic Bayesian Active Learning for Linear Regression
Stephen Mussmann
Active learning studies the fundamental question: what data should we choose to observe? The greedy algorithm in optimal experiment design is a common heuristic and also equivalent…
Instance-Level Costs for Nuanced Classifier Evaluation
Kabir Kang, Stephen Mussmann
Standard classification treats all errors equally, but in applications such as content moderation and medical screening, mistakes on clear-cut cases are more costly than errors on…
Sum Estimation via Vector Similarity Search
Stephen Mussmann, Mehul Smriti Raje, Kavya Tumkur +3
Semantic embeddings to represent objects such as image, text and audio are widely used in machine learning and have spurred the development of vector similarity search methods for…
Batch Bayesian Active Learning with Partial Batch Label Sampling
Kangping Hu, Stephen Mussmann
Over the past couple of decades, many active learning acquisition functions have been proposed, leaving practitioners with an unclear choice of which to use. Bayesian active learni…
Continuous nonlinear adaptive experimental design with gradient flow
Ruhui Jin, Qin Li, Stephen O. Mussmann +1
In computational inverse problems, the optimal experimental design (OED) problem seeks the best locations in time and space at which to take measurements. We investigate the nonlin…
An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
Gantavya Bhatt, Yifang Chen, Arnav M. Das +9
Supervised finetuning (SFT) on instruction datasets has played a crucial role in achieving the remarkable zero-shot generalization capabilities observed in modern large language mo…
LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning
Jifan Zhang, Yifang Chen, Gregory Canal +8
Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive. To mitigate this cost, machine learning methods, such as transfer learning…
DataComp: In search of the next generation of multimodal datasets
Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang +31
Multimodal datasets are a critical component in recent breakthroughs such as Stable Diffusion and GPT-4, yet their design does not receive the same research attention as model arch…
VOCALExplore: Pay-as-You-Go Video Data Exploration and Model Building [Technical Report]
Maureen Daum, Enhao Zhang, Dong He +4
We introduce VOCALExplore, a system designed to support users in building domain-specific models over video datasets. VOCALExplore supports interactive labeling sessions and trains…
Active Learning with Expected Error Reduction
Stephen Mussmann, Julia Reisler, Daniel Tsai +3
Active learning has been studied extensively as a method for efficient data collection. Among the many approaches in literature, Expected Error Reduction (EER) (Roy and McCallum) h…
Comparing the Value of Labeled and Unlabeled Data in Method-of-Moments Latent Variable Estimation
Mayee F. Chen, Benjamin Cohen-Wang, Stephen Mussmann +2
Labeling data for modern machine learning is expensive and time-consuming. Latent variable models can be used to infer labels from weaker, easier-to-acquire sources operating on un…
On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks
Stephen Mussmann, Robin Jia, Percy Liang
Many pairwise classification tasks, such as paraphrase detection and open-domain question answering, naturally have extreme label imbalance (e.g., of examples are negativ…
Concept Bottleneck Models
Pang Wei Koh, Thao Nguyen, Yew Siang Tang +4
We seek to learn models that we can interact with using high-level concepts: if the model did not think there was a bone spur in the x-ray, would it still predict severe arthritis?…
Selection via Proxy: Efficient Data Selection for Deep Learning
Cody Coleman, Christopher Yeh, Stephen Mussmann +5
Data selection methods, such as active learning and core-set selection, are useful tools for machine learning on large datasets. However, they can be prohibitively expensive to app…
A Tight Analysis of Greedy Yields Subexponential Time Approximation for Uniform Decision Tree
Ray Li, Percy Liang, Stephen Mussmann
Decision Tree is a classic formulation of active learning: given hypotheses with nonnegative weights summing to 1 and a set of tests that each partition the hypotheses, output…
Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss
Stephen Mussmann, Percy Liang
Uncertainty sampling, a popular active learning algorithm, is used to reduce the amount of data required to learn a classifier, but it has been observed in practice to converge to…
The price of debiasing automatic metrics in natural language evaluation
Arun Tejasvi Chaganty, Stephen Mussman, Percy Liang
For evaluating generation systems, automatic metrics such as BLEU cost nothing to run but have been shown to correlate poorly with human judgment, leading to systematic bias agains…
On the Relationship between Data Efficiency and Error for Uncertainty Sampling
Stephen Mussmann, Percy Liang
While active learning offers potential cost savings, the actual data efficiency---the reduction in amount of labeled data needed to obtain the same error rate---observed in practic…
Generalized Binary Search For Split-Neighborly Problems
Stephen Mussmann, Percy Liang
In sequential hypothesis testing, Generalized Binary Search (GBS) greedily chooses the test with the highest information gain at each step. It is known that GBS obtains the gold st…
Fast Amortized Inference and Learning in Log-linear Models with Randomly Perturbed Nearest Neighbor Search
Stephen Mussmann, Daniel Levy, Stefano Ermon
Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks wh…
Understanding Trajectory Behavior: A Motion Pattern Approach
Mahdi M. Kalayeh, Stephen Mussmann, Alla Petrakova +2
Mining the underlying patterns in gigantic and complex data is of great importance to data analysts. In this paper, we propose a motion pattern approach to mine frequent behaviors…