4 papers
Stochastically Dominant Peer Prediction
Yichi Zhang, Shengwei Xu, David Pennock +1
Eliciting reliable human feedback is essential for many machine learning tasks, such as learning from noisy labels and aligning AI systems with human preferences. Peer prediction m…
Eliciting Informative Text Evaluations with Large Language Models
Yuxuan Lu, Shengwei Xu, Yichi Zhang +2
Peer prediction mechanisms motivate high-quality feedback with provable guarantees. However, current methods only apply to rather simple reports, like multiple-choice or scalar num…
Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms
Shengwei Xu, Yichi Zhang, Paul Resnick +1
Because high-quality data is like oxygen for AI systems, effectively eliciting information from crowdsourcing workers has become a first-order problem for developing high-performan…
Eliciting Honest Information From Authors Using Sequential Review
Yichi Zhang, Grant Schoenebeck, Weijie Su
In the setting of conference peer review, the conference aims to accept high-quality papers and reject low-quality papers based on noisy review scores. A recent work proposes the i…