15 citations · 23 across the 10 of their papers we have counts for
12 papers
Mutual Evaluation and Supervision without Peers
Zachary Robertson
This article introduces mutual evaluation of a replicable task worker and a critic that incentivizes truthful reporting, both modeled as strategic agents. The critic chooses a fini…
A Global Characterization of -Divergences Yielding PSD Mutual-Information Matrices
Zachary Robertson
Given random variables, when does the matrix of pairwise -mutual informations define a PSD kernel over variables? For convex finite generators wi…
Identity-Link IRT for Label-Free LLM Evaluation: Preserving Additivity in TVD-MI Scores
Zachary Robertson
Pairwise comparisons of large language models using total variation distance mutual information (TVD-MI) produce binary critic decisions per pair. We show that averaging TVD-MI's b…
Value Alignment of Social Media Ranking Algorithms
Farnaz Jahanbakhsh, Dora Zhao, Tiziano Piccardi +4
While social media feed rankings are primarily driven by engagement signals rather than any explicit value system, the resulting algorithmic feeds are not value-neutral: engagement…
Let's Measure Information Step-by-Step: AI-Based Evaluation Beyond Vibes
Zachary Robertson, Sanmi Koyejo
We evaluate artificial intelligence (AI) systems without ground truth by exploiting a link between strategic gaming and information loss. Building on established information theory…
Measurement to Meaning: A Validity-Centered Framework for AI Evaluation
Olawale Salaudeen, Anka Reuel, Ahmed Ahmed +6
While the capabilities and utility of AI systems have advanced, rigorous norms for evaluating these systems have lagged. Grand claims, such as models achieving general reasoning ca…
GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Study
Zachary Robertson
In this pilot study, we investigate the use of GPT4 to assist in the peer-review process. Our key hypothesis was that GPT-generated reviews could achieve comparable helpfulness to…
No Bidding, No Regret: Pairwise-Feedback Mechanisms for Digital Goods and Data Auctions
Zachary Robertson, Oluwasanmi Koyejo
The growing demand for data and AI-generated digital goods, such as personalized written content and artwork, necessitates effective pricing and feedback mechanisms that account fo…
Implicit Regularization in Feedback Alignment Learning Mechanisms for Neural Networks
Zachary Robertson, Oluwasanmi Koyejo
Feedback Alignment (FA) methods are biologically inspired local learning rules for training neural networks with reduced communication between layers. While FA has potential applic…
Pairwise Ranking Losses of Click-Through Rates Prediction for Welfare Maximization in Ad Auctions
Boxiang Lyu, Zhe Feng, Zachary Robertson +1
We study the design of loss functions for click-through rates (CTR) to optimize (social) welfare in advertising auctions. Existing works either only focus on CTR predictions withou…
Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle
Rylan Schaeffer, Mikail Khona, Zachary Robertson +5
Double descent is a surprising phenomenon in machine learning, in which as the number of model parameters grows relative to the number of data, test error drops as models grow ever…
Concurrent Training Improves the Performance of Behavioral Cloning from Observation
Zachary W. Robertson, Matthew R. Walter
Learning from demonstration is widely used as an efficient way for robots to acquire new skills. However, it typically requires that demonstrations provide full access to the state…