1.4k citations · 5k across the 110 of their papers we have counts for
112 papers
What Should Data Science Education Do with Large Language Models?
Xinming Tu, James Zou, Weijie J. Su +1
The rapid advances of large language models (LLMs), such as ChatGPT, are revolutionizing data science and statistics. These state-of-the-art tools can streamline complex processes.…
How is ChatGPT's behavior changing over time?
Lingjiao Chen, Matei Zaharia, James Zou
GPT-3.5 and GPT-4 are the two most widely used large language model (LLM) services. However, when and how these models are updated over time is opaque. Here, we evaluate the March…
Discover and Cure: Concept-aware Mitigation of Spurious Correlation
Shirley Wu, Mert Yuksekgonul, Linjun Zhang +1
Deep neural networks often rely on spurious correlations to make predictions, which hinders generalization beyond training environments. For instance, models that associate cats wi…
OpenDataVal: a Unified Benchmark for Data Valuation
Kevin Fu Jiang, Weixin Liang, James Zou +1
Assessing the quality and impact of individual data points is critical for improving model performance and mitigating undesirable biases within the training dataset. Several data v…
TWIGMA: A dataset of AI-Generated Images with Metadata From Twitter
Yiqun Chen, James Zou
Recent progress in generative artificial intelligence (gen-AI) has enabled the generation of photo-realistic and artistically-inspiring photos at a single click, catering to millio…
ArtWhisperer: A Dataset for Characterizing Human-AI Interactions in Artistic Creations
Kailas Vodrahalli, James Zou
As generative AI becomes more prevalent, it is important to study how human users interact with such models. In this work, we investigate how people use text-to-image models to gen…
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy
Paul Pu Liang, Zihao Deng, Martin Ma +3
In a wide range of multimodal tasks, contrastive learning has become a particularly appealing approach since it can successfully learn representations from abundant unlabeled data…
Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value
Yongchan Kwon, James Zou
Data valuation is a powerful framework for providing statistical insights into which data are beneficial or detrimental to model training. Many Shapley-based data valuation methods…
Accuracy on the Curve: On the Nonlinear Correlation of ML Performance Between Data Subpopulations
Weixin Liang, Yining Mao, Yongchan Kwon +2
Understanding the performance of machine learning (ML) models across diverse data distributions is critically important for reliable applications. Despite recent empirical studies…
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou +4
Language models have been shown to exhibit positive scaling, where performance improves as models are scaled up in terms of size, compute, or data. In this work, we introduce NeQA,…
FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
Lingjiao Chen, Matei Zaharia, James Zou
There is a rapidly growing number of large language models (LLMs) that users can query for a fee. We review the cost associated with querying popular LLM APIs, e.g. GPT-4, ChatGPT,…
Beyond Confidence: Reliable Models Should Also Consider Atypicality
Mert Yuksekgonul, Linjun Zhang, James Zou +1
While most machine learning models can provide confidence in their predictions, confidence is insufficient to understand a prediction's reliability. For instance, the model may hav…
Data-Driven Subgroup Identification for Linear Regression
Zachary Izzo, Ruishan Liu, James Zou
Medical studies frequently require to extract the relationship between each covariate and the outcome with statistical confidence measures. To do this, simple parametric models are…
Last-Layer Fairness Fine-tuning is Simple and Effective for Neural Networks
Yuzhen Mao, Zhun Deng, Huaxiu Yao +3
As machine learning has been deployed ubiquitously across applications in modern data science, algorithmic fairness has become a great concern. Among them, imposing fairness constr…
GPT detectors are biased against non-native English writers
Weixin Liang, Mert Yuksekgonul, Yining Mao +2
The rapid adoption of generative language models has brought about substantial advancements in digital communication, while simultaneously raising concerns regarding the potential…
Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data
Ryumei Nakada, Halil Ibrahim Gulluk, Zhun Deng +3
Language-supervised vision models have recently attracted great attention in computer vision. A common approach to build such models is to use contrastive learning on paired data a…
Diagnosing and Rectifying Vision Models using Language
Yuhui Zhang, Jeff Z. HaoChen, Shih-Cheng Huang +3
Recent multi-modal contrastive learning models have demonstrated the ability to learn an embedding space suitable for building strong vision classifiers, by leveraging the rich inf…
Provable Membership Inference Privacy
Zachary Izzo, Jinsung Yoon, Sercan O. Arik +1
In applications involving sensitive data, such as finance and healthcare, the necessity for preserving data privacy can be a significant barrier to machine learning model developme…
Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale
Federico Bianchi, Pratyusha Kalluri, Esin Durmus +7
Machine learning models that convert user-written text descriptions into images are now widely available online and used by millions of users to generate millions of images a day.…
A Spectral Method for Assessing and Combining Multiple Data Visualizations
Rong Ma, Eric D. Sun, James Zou
Dimension reduction and data visualization aim to project a high-dimensional dataset to a low-dimensional space while capturing the intrinsic structures in the data. It is an indis…
SEAL : Interactive Tool for Systematic Error Analysis and Labeling
Nazneen Rajani, Weixin Liang, Lingjiao Chen +2
With the advent of Transformers, large language models (LLMs) have saturated well-known NLP benchmarks and leaderboards with high aggregate performance. However, many times these m…
C-Mixup: Improving Generalization in Regression
Huaxiu Yao, Yiping Wang, Linjun Zhang +2
Improving the generalization of deep networks is an important open challenge, particularly in domains without plentiful data. The mixup algorithm improves generalization by linearl…
Knowledge-Driven New Drug Recommendation
Zhenbang Wu, Huaxiu Yao, Zhe Su +5
Drug recommendation assists doctors in prescribing personalized medications to patients based on their health conditions. Existing drug recommendation solutions adopt the supervise…
Data Budgeting for Machine Learning
Xinyi Zhao, Weixin Liang, James Zou
Data is the fuel powering AI and creates tremendous value for many domains. However, collecting datasets for AI is a time-consuming, expensive, and complicated endeavor. For practi…