1.4k citations · 5k across the 110 of their papers we have counts for
19 papers · 1 filter
How is ChatGPT's behavior changing over time?
Lingjiao Chen, Matei Zaharia, James Zou
GPT-3.5 and GPT-4 are the two most widely used large language model (LLM) services. However, when and how these models are updated over time is opaque. Here, we evaluate the March…
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou +4
Language models have been shown to exhibit positive scaling, where performance improves as models are scaled up in terms of size, compute, or data. In this work, we introduce NeQA,…
GPT detectors are biased against non-native English writers
Weixin Liang, Mert Yuksekgonul, Yining Mao +2
The rapid adoption of generative language models has brought about substantial advancements in digital communication, while simultaneously raising concerns regarding the potential…
Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale
Federico Bianchi, Pratyusha Kalluri, Esin Durmus +7
Machine learning models that convert user-written text descriptions into images are now widely available online and used by millions of users to generate millions of images a day.…
SEAL : Interactive Tool for Systematic Error Analysis and Labeling
Nazneen Rajani, Weixin Liang, Lingjiao Chen +2
With the advent of Transformers, large language models (LLMs) have saturated well-known NLP benchmarks and leaderboards with high aggregate performance. However, many times these m…
GSCLIP : A Framework for Explaining Distribution Shifts in Natural Language
Zhiying Zhu, Weixin Liang, James Zou
Helping end users comprehend the abstract distribution shifts can greatly facilitate AI deployment. Motivated by this, we propose a novel task, dataset explanation. Given two image…
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
Weixin Liang, Yuhui Zhang, Yongchan Kwon +2
We present modality gap, an intriguing geometric phenomenon of the representation space of multi-modal models. Specifically, we show that different data modalities (e.g. images and…
Persistent Anti-Muslim Bias in Large Language Models
Abubakar Abid, Maheen Farooqi, James Zou
It has been observed that large-scale language models capture undesirable societal biases, e.g. relating to race and gender; yet religious bias has been relatively unexplored. We d…
ALICE: Active Learning with Contrastive Natural Language Explanations
Weixin Liang, James Zou, Zhou Yu
Training a supervised neural network classifier typically requires many annotated training samples. Collecting and annotating a large number of data points are costly and sometimes…
Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation
Weixin Liang, James Zou, Zhou Yu
Open Domain dialog system evaluation is one of the most important challenges in dialog research. Existing automatic evaluation metrics, such as BLEU are mostly reference-based. The…
LitGen: Genetic Literature Recommendation Guided by Human Explanations
Allen Nie, Arturo L. Pineda, Matt W. Wright Hannah Wand +5
As genetic sequencing costs decrease, the lack of clinical interpretation of variants has become the bottleneck in using genetics data. A major rate limiting step in clinical inter…
Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootings
Dorottya Demszky, Nikhil Garg, Rob Voigt +4
We provide an NLP framework to uncover four linguistic dimensions of political polarization in social media: topic choice, framing, affect and illocutionary force. We quantify thes…
Large-scale Generative Modeling to Improve Automated Veterinary Disease Coding
Yuhui Zhang, Allen Nie, James Zou
Supervised learning is limited both by the quantity and quality of the labeled data. In the field of medical record tagging, writing styles between hospitals vary drastically. The…
DeepTag: inferring all-cause diagnoses from clinical notes in under-resourced medical domain
Allen Nie, Ashley Zehnder, Rodney L. Page +4
Large scale veterinary clinical records can become a powerful resource for patient care and research. However, clinicians lack the time and resource to annotate patient records wit…
CoVeR: Learning Covariate-Specific Vector Representations with Tensor Decompositions
Kevin Tian, Teng Zhang, James Zou
Word embedding is a useful approach to capture co-occurrence structures in large text corpora. However, in addition to the text data itself, we often have additional covariates ass…
Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
Nikhil Garg, Londa Schiebinger, Dan Jurafsky +1
Word embeddings use vectors to represent words such that the geometry between vectors captures semantic relationship between the words. In this paper, we develop a framework to dem…
Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
Shyam Upadhyay, Kai-Wei Chang, Matt Taddy +2
Word embeddings, which represent a word as a point in a vector space, have become ubiquitous to several NLP tasks. A recent line of work uses bilingual (two languages) corpora to l…
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Tolga Bolukbasi, Kai-Wei Chang, James Zou +2
The blind application of machine learning runs the risk of amplifying biases present in data. Such a danger is facing us with word embedding, a popular framework to represent text…
Quantifying and Reducing Stereotypes in Word Embeddings
Tolga Bolukbasi, Kai-Wei Chang, James Zou +2
Machine learning algorithms are optimized to model statistical properties of the training data. If the input data reflects stereotypes and biases of the broader society, then the o…