7 papers
Aggregated Individual Reporting for Post-Deployment Evaluation
Jessica Dai, Inioluwa Deborah Raji, Benjamin Recht +1
The need for developing model evaluations beyond static benchmarking, especially in the post-deployment phase, is now well-understood. At the same time, concerns about the concentr…
Three Years of r/ChatGPT: Societal Impact Evaluations from Social Media Data
Jessica Dai, Sean Garcia, Emma Pierson +2
ChatGPT was launched on November 30, 2022; the r/ChatGPT subreddit was created just one day later. Since then, chatbot-based AI products have gone from niche proofs-of-concept to w…
Separating Geometry from Probability in the Analysis of Generalization
Maxim Raginsky, Benjamin Recht
The goal of machine learning is to find models that minimize prediction error on data that has not yet been seen. Its operational paradigm assumes access to a dataset and artic…
Gradient Descent Provably Solves Nonlinear Tomographic Reconstruction
Sara Fridovich-Keil, Fabrizio Valdivia, Gordon Wetzstein +2
In computed tomography (CT), the forward model consists of a linear Radon transform followed by an exponential nonlinearity based on the attenuation of light according to the Beer-…
Bridging Prediction and Intervention Problems in Social Systems
Lydia T. Liu, Inioluwa Deborah Raji, Angela Zhou +32
Many automated decision systems (ADS) are designed to solve prediction problems -- where the goal is to learn patterns from a sample of the population and apply them to individuals…
In Defense of Defensive Forecasting
Juan Carlos Perdomo, Benjamin Recht
This tutorial provides a survey of algorithms for Defensive Forecasting, where predictions are derived not by prognostication but by correcting past mistakes. Pioneered by Vovk, De…