85 citations · 200 across the 24 of their papers we have counts for
24 papers
Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking
Benjamin Feuer, Micah Goldblum, Teresa Datta +5
The release of ChatGPT in November 2022 sparked an explosion of interest in post-training and an avalanche of new preference optimization (PO) methods. These methods claim superior…
Large Language Models Must Be Taught to Know What They Don't Know
Sanyam Kapoor, Nate Gruver, Manley Roberts +7
When using large language models (LLMs) in high-stakes applications, we need to know when we can trust their predictions. Some works argue that prompting high-performance LLMs is s…
Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
Arka Pal, Deep Karkhanis, Samuel Dooley +3
Direct Preference Optimisation (DPO) is effective at significantly improving the performance of large language models (LLMs) on downstream tasks such as reasoning, summarisation, a…
ForecastPFN: Synthetically-Trained Zero-Shot Forecasting
Samuel Dooley, Gurnoor Singh Khurana, Chirag Mohapatra +2
The vast majority of time-series forecasting approaches require a substantial training dataset. However, many real-life forecasting applications have very little initial observatio…
Fairer and More Accurate Tabular Models Through NAS
Richeek Das, Samuel Dooley
Making models algorithmically fairer in tabular data has been long studied, with techniques typically oriented towards fixes which usually take a neural model with an undesirable o…
Data Contamination Through the Lens of Time
Manley Roberts, Himanshu Thakur, Christine Herlihy +2
Recent claims about the impressive abilities of large language models (LLMs) are often supported by evaluating publicly available benchmarks. Since LLMs train on wide swaths of the…
Giraffe: Adventures in Expanding Context Lengths in LLMs
Arka Pal, Deep Karkhanis, Manley Roberts +3
Modern large language models (LLMs) that rely on attention mechanisms are typically trained with fixed context lengths which enforce upper limits on the length of input sequences t…
Robustness Disparities in Face Detection
Samuel Dooley, George Z. Wei, Tom Goldstein +1
Facial analysis systems have been deployed by large companies and critiqued by scholars and activists for the past decade. Many existing algorithmic audits examine the performance…
How Technology Impacts and Compares to Humans in Socially Consequential Arenas
Samuel Dooley
One of the main promises of technology development is for it to be adopted by people, organizations, societies, and governments -- incorporated into their life, work stream, or pro…
Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition
Samuel Dooley, Rhea Sanjay Sukthanker, John P. Dickerson +3
Face recognition systems are widely deployed in safety-critical applications, including law enforcement, yet they exhibit bias across a range of socio-demographic dimensions, such…
A Deep Dive into Dataset Imbalance and Bias in Face Identification
Valeriia Cherepanova, Steven Reich, Samuel Dooley +3
As the deployment of automated face recognition (FR) systems proliferates, bias in these systems is not just an academic question, but a matter of public concern. Media portrayals…
Topological Data Analysis for Word Sense Disambiguation
Michael Rawson, Samuel Dooley, Mithun Bharadwaj +1
We develop and test a novel unsupervised algorithm for word sense induction and disambiguation which uses topological data analysis. Typical approaches to the problem involve clust…
The Dichotomous Affiliate Stable Matching Problem: Approval-Based Matching with Applicant-Employer Relations
Marina Knittel, Samuel Dooley, John P. Dickerson
While the stable marriage problem and its variants model a vast range of matching markets, they fail to capture complex agent relationships, such as the affiliation of applicants a…
Are Commercial Face Detection Models as Biased as Academic Models?
Samuel Dooley, George Z. Wei, Tom Goldstein +1
As facial recognition systems are deployed more widely, scholars and activists have studied their biases and harms. Audits are commonly used to accomplish this and compare the algo…
Comparing Human and Machine Bias in Face Recognition
Samuel Dooley, Ryan Downing, George Wei +10
Much recent research has uncovered and discussed serious concerns of bias in facial analysis technologies, finding performance disparities between groups of people based on perceiv…
Ctrl-Shift: How Privacy Sentiment Changed from 2019 to 2021
Angelica Goetzen, Samuel Dooley, Elissa M. Redmiles
People's privacy sentiments influence changes in legislation as well as technology design and use. While single-point-in-time investigations of privacy sentiment offer useful insig…
Robustness Disparities in Commercial Face Detection
Samuel Dooley, Tom Goldstein, John P. Dickerson
Facial detection and analysis systems have been deployed by large companies and critiqued by scholars and activists for the past decade. Critiques that focus on system performance…
PreferenceNet: Encoding Human Preferences in Auction Design with Deep Learning
Neehar Peri, Michael J. Curry, Samuel Dooley +1
The design of optimal auctions is a problem of interest in economics, game theory and computer science. Despite decades of effort, strategyproof, revenue-maximizing auction designs…
ProportionNet: Balancing Fairness and Revenue for Auction Design with Deep Learning
Kevin Kuo, Anthony Ostuni, Elizabeth Horishny +5
The design of revenue-maximizing auctions with strong incentive guarantees is a core concern of economic theory. Computational auctions enable online advertising, sourcing, spectru…
The Affiliate Matching Problem: On Labor Markets where Firms are Also Interested in the Placement of Previous Workers
Samuel Dooley, John P. Dickerson
In many labor markets, workers and firms are connected via affiliative relationships. A management consulting firm wishes to both accept the best new workers but also place its cur…
Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning
Vedant Nanda, Samuel Dooley, Sahil Singla +2
Deep neural networks (DNNs) are increasingly used in real-world applications (e.g. facial recognition). This has resulted in concerns about the fairness of decisions made by these…
Can an Algorithm be My Healthcare Proxy?
Duncan C McElfresh, Samuel Dooley, Yuan Cui +5
Planning for death is not a process in which everyone participates. Yet a lack of planning can have vast impacts on a patient's well-being, the well-being of her family, and the me…
Overhead Detection: Beyond 8-bits and RGB
Eliza Mace, Keith Manville, Monica Barbu-McInnis +3
This study uses the challenging and publicly available SpaceNet dataset to establish a performance baseline for a state-of-the-art object detector in satellite imagery. Specificall…
xView: Objects in Context in Overhead Imagery
Darius Lam, Richard Kuzma, Kevin McGee +5
We introduce a new large-scale dataset for the advancement of object detection techniques and overhead object detection research. This satellite imagery dataset enables research pr…