1.4k citations · 5k across the 110 of their papers we have counts for
60 papers · 1 filter
Discover and Cure: Concept-aware Mitigation of Spurious Correlation
Shirley Wu, Mert Yuksekgonul, Linjun Zhang +1
Deep neural networks often rely on spurious correlations to make predictions, which hinders generalization beyond training environments. For instance, models that associate cats wi…
OpenDataVal: a Unified Benchmark for Data Valuation
Kevin Fu Jiang, Weixin Liang, James Zou +1
Assessing the quality and impact of individual data points is critical for improving model performance and mitigating undesirable biases within the training dataset. Several data v…
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy
Paul Pu Liang, Zihao Deng, Martin Ma +3
In a wide range of multimodal tasks, contrastive learning has become a particularly appealing approach since it can successfully learn representations from abundant unlabeled data…
Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value
Yongchan Kwon, James Zou
Data valuation is a powerful framework for providing statistical insights into which data are beneficial or detrimental to model training. Many Shapley-based data valuation methods…
Accuracy on the Curve: On the Nonlinear Correlation of ML Performance Between Data Subpopulations
Weixin Liang, Yining Mao, Yongchan Kwon +2
Understanding the performance of machine learning (ML) models across diverse data distributions is critically important for reliable applications. Despite recent empirical studies…
FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
Lingjiao Chen, Matei Zaharia, James Zou
There is a rapidly growing number of large language models (LLMs) that users can query for a fee. We review the cost associated with querying popular LLM APIs, e.g. GPT-4, ChatGPT,…
Beyond Confidence: Reliable Models Should Also Consider Atypicality
Mert Yuksekgonul, Linjun Zhang, James Zou +1
While most machine learning models can provide confidence in their predictions, confidence is insufficient to understand a prediction's reliability. For instance, the model may hav…
Data-Driven Subgroup Identification for Linear Regression
Zachary Izzo, Ruishan Liu, James Zou
Medical studies frequently require to extract the relationship between each covariate and the outcome with statistical confidence measures. To do this, simple parametric models are…
Last-Layer Fairness Fine-tuning is Simple and Effective for Neural Networks
Yuzhen Mao, Zhun Deng, Huaxiu Yao +3
As machine learning has been deployed ubiquitously across applications in modern data science, algorithmic fairness has become a great concern. Among them, imposing fairness constr…
Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data
Ryumei Nakada, Halil Ibrahim Gulluk, Zhun Deng +3
Language-supervised vision models have recently attracted great attention in computer vision. A common approach to build such models is to use contrastive learning on paired data a…
Diagnosing and Rectifying Vision Models using Language
Yuhui Zhang, Jeff Z. HaoChen, Shih-Cheng Huang +3
Recent multi-modal contrastive learning models have demonstrated the ability to learn an embedding space suitable for building strong vision classifiers, by leveraging the rich inf…
Provable Membership Inference Privacy
Zachary Izzo, Jinsung Yoon, Sercan O. Arik +1
In applications involving sensitive data, such as finance and healthcare, the necessity for preserving data privacy can be a significant barrier to machine learning model developme…
C-Mixup: Improving Generalization in Regression
Huaxiu Yao, Yiping Wang, Linjun Zhang +2
Improving the generalization of deep networks is an important open challenge, particularly in domains without plentiful data. The mixup algorithm improves generalization by linearl…
Knowledge-Driven New Drug Recommendation
Zhenbang Wu, Huaxiu Yao, Zhe Su +5
Drug recommendation assists doctors in prescribing personalized medications to patients based on their health conditions. Existing drug recommendation solutions adopt the supervise…
Data Budgeting for Machine Learning
Xinyi Zhao, Weixin Liang, James Zou
Data is the fuel powering AI and creates tremendous value for many domains. However, collecting datasets for AI is a time-consuming, expensive, and complicated endeavor. For practi…
Ensembling improves stability and power of feature selection for deep learning models
Prashnna K Gyawali, Xiaoxia Liu, James Zou +1
With the growing adoption of deep learning models in different real-world domains, including computational biology, it is often necessary to understand which data features are esse…
Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise
Haotian Ye, James Zou, Linjun Zhang
The existence of spurious correlations such as image backgrounds in the training environment can make empirical risk minimization (ERM) perform badly in the test environment. To ad…
DataPerf: Benchmarks for Data-Centric AI Development
Mark Mazumder, Colby Banbury, Xiaozhe Yao +42
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithf…
FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Data
Zhun Deng, Jiayao Zhang, Linjun Zhang +4
Algorithmic fairness plays an important role in machine learning and imposing fairness constraints during learning is a common approach. However, many datasets are imbalanced in ce…
Post-hoc Concept Bottleneck Models
Mert Yuksekgonul, Maggie Wang, James Zou
Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable concepts (``the bottleneck'') and use the concepts to make predictions. A concept bottleneck enhances in…
Domino: Discovering Systematic Errors with Cross-Modal Embeddings
Sabri Eyuboglu, Maya Varma, Khaled Saab +5
Machine learning models that achieve high overall accuracy often make systematic errors on important subsets (or slices) of data. Identifying underperforming slices is particularly…
MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflicts
Weixin Liang, James Zou
Understanding the performance of machine learning models across diverse data distributions is critically important for reliable applications. Motivated by this, there is a growing…
Submix: Practical Private Prediction for Large-Scale Language Models
Antonio Ginart, Laurens van der Maaten, James Zou +1
Recent data-extraction attacks have exposed that language models can memorize some training samples verbatim. This is a vulnerability that can compromise the privacy of the model's…
Competition over data: how does data purchase affect users?
Yongchan Kwon, Antonio Ginart, James Zou
As machine learning (ML) is deployed by many competing service providers, the underlying ML predictors also compete against each other, and it is increasingly important to understa…