7 papers
Distributed Denial of Science: How Indirect Data Poisoning of AI Systems Can Industrialize Scientific Fraud
Bálint Gyevnár, Atoosa Kasirzadeh, Nihar B. Shah
Scientific fraud is the instrument of doubt that malicious entities can use to establish controversy in science. Historically, it required the resources of a company: deep pockets,…
The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems
Ziming Luo, Atoosa Kasirzadeh, Nihar B. Shah
AI scientist systems, capable of autonomously executing the full research workflow from hypothesis generation and experimentation to paper writing, hold significant potential for a…
ML Researchers Support Openness in Peer Review But Are Concerned About Resubmission Bias
Vishisht Rao, Justin Payan, Andrew McCallum +1
Peer-review venues have increasingly adopted open reviewing policies that publicly release anonymized reviews and permit public commenting. Venues have adopted a variety of policie…
FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers
Sarina Xi, Vishisht Rao, Justin Payan +1
The identification and localization of errors is a core task in peer review, yet the exponential growth of scientific output has made it increasingly difficult for human reviewers…
Who is a Better Matchmaker? Human vs. Algorithmic Judge Assignment in a High-Stakes Startup Competition
Sarina Xi, Orelia Pi, Miaomiao Zhang +3
There is growing interest in applying artificial intelligence (AI) to automate and support complex decision-making tasks. However, it remains unclear how algorithms compare to huma…
Designing Rules to Pick a Rule: Aggregation by Consistency
Ratip Emin Berker, Ben Armstrong, Vincent Conitzer +1
Given a set of items and a set of evaluators who all individually rank them, how do we aggregate these evaluations into a single societal ranking? Work in social choice and statist…