134 citations · 262 across the 17 of their papers we have counts for
21 papers
Societal Impacts Research Requires Benchmarks for Creative Composition Tasks
Judy Hanwen Shen, Carlos Guestrin
Foundation models that are capable of automating cognitive tasks represent a pivotal technological shift, yet their societal implications remain unclear. These systems promise exci…
Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Lee Cohen, Jack Hsieh, Connie Hong +1
In an era of increasingly capable foundation models, job seekers are turning to generative AI tools to enhance their application materials. However, unequal access to and knowledge…
Algorithms with Calibrated Machine Learning Predictions
Judy Hanwen Shen, Ellen Vitercik, Anders Wikum
The field of algorithms with predictions incorporates machine learning advice in the design of online algorithms to improve real-world performance. A central consideration is the e…
Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparison
Judy Hanwen Shen, Archit Sharma, Jun Qin
The goal of aligning language models to human preferences requires data that reveal these preferences. Ideally, time and money can be spent carefully collecting and tailoring bespo…
The Data Addition Dilemma
Judy Hanwen Shen, Inioluwa Deborah Raji, Irene Y. Chen
In many machine learning for healthcare tasks, standard datasets are constructed by amassing data across many, often fundamentally dissimilar, sources. But when does adding more da…
Multigroup Robustness
Lunjia Hu, Charlotte Peale, Judy Hanwen Shen
To address the shortcomings of real-world datasets, robust learning algorithms have been designed to overcome arbitrary and indiscriminate data corruption. However, practical proce…
Unifying Corroborative and Contributive Attributions in Large Language Models
Theodora Worledge, Judy Hanwen Shen, Nicole Meister +2
As businesses, products, and services spring up around large language models, the trustworthiness of these models hinges on the verifiability of their outputs. However, methods for…
Unlocking Accuracy and Fairness in Differentially Private Image Classification
Leonard Berrada, Soham De, Judy Hanwen Shen +6
Privacy-preserving machine learning aims to train models on private data without leaking sensitive information. Differential privacy (DP) is considered the gold standard framework…
Dissenting Explanations: Leveraging Disagreement to Reduce Model Overreliance
Omer Reingold, Judy Hanwen Shen, Aditi Talati
While explainability is a desirable characteristic of increasingly complex black-box models, modern explanation methods have been shown to be inconsistent and contradictory. The se…
Bidding Strategies for Proportional Representation in Advertisement Campaigns
Inbal Livni Navon, Charlotte Peale, Omer Reingold +1
Many companies rely on advertising platforms such as Google, Facebook, or Instagram to recruit a large and diverse applicant pool for job openings. Prior works have shown that equi…
Leximax Approximations and Representative Cohort Selection
Monika Henzinger, Charlotte Peale, Omer Reingold +1
Finding a representative cohort from a broad pool of candidates is a goal that arises in many contexts such as choosing governing committees and consumer panels. While there are ma…
Longitudinal Complex Dynamics of Labour Markets Reveal Increasing Polarisation
Shahad Althobaiti, Ahmad Alabdulkareem, Judy Hanwen Shen +4
In this paper we conduct a longitudinal analysis of the structure of labour markets in the US over 7 decades of technological, economic and policy change. We make use of network sc…
Accuracy, Interpretability, and Differential Privacy via Explainable Boosting
Harsha Nori, Rich Caruana, Zhiqi Bu +2
We show that adding differential privacy to Explainable Boosting Machines (EBMs), a recent method for training interpretable ML models, yields state-of-the-art accuracy while prote…
Fast and Memory Efficient Differentially Private-SGD via JL Projections
Zhiqi Bu, Sivakanth Gopi, Janardhan Kulkarni +3
Differentially Private-SGD (DP-SGD) of Abadi et al. (2016) and its variations are the only known algorithms for private training of large scale neural networks. This algorithm requ…
Human-centric Dialog Training via Offline Reinforcement Learning
Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun +5
How can we train a dialog model to produce better conversations by learning from human feedback, without the risk of humans teaching it harmful chat behaviors? We start by hosting…
Differentially Private Set Union
Sivakanth Gopi, Pankaj Gulhane, Janardhan Kulkarni +3
We study the basic operation of set union in the global model of differential privacy. In this problem, we are given a universe of items, possibly of infinite size, and a datab…
Hierarchical Reinforcement Learning for Open-Domain Dialog
Abdelrhman Saleh, Natasha Jaques, Asma Ghandeharioun +2
Open-domain dialog generation is a challenging problem; maximum likelihood training can lead to repetitive outputs, models have difficulty tracking long-term conversational goals,…
Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen +5
Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. These are critica…
Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
Asma Ghandeharioun, Judy Hanwen Shen, Natasha Jaques +4
Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation qua…
Comparing Models of Associative Meaning: An Empirical Investigation of Reference in Simple Language Games
Judy Hanwen Shen, Matthias Hofer, Bjarke Felbo +1
Simple reference games are of central theoretical and empirical importance in the study of situated language use. Although language provides rich, compositional truth-conditional s…
Closing the AI Knowledge Gap
Ziv Epstein, Blakeley H. Payne, Judy Hanwen Shen +6
AI researchers employ not only the scientific method, but also methodology from mathematics and engineering. However, the use of the scientific method - specifically hypothesis tes…