4 papers
greCAPTCHA: Assessing Understanding as Evidence of Research Authorship Under Generative AI
Justin Payan, Bálint Gyevnár, Atoosa Kasirzadeh +1
Conferences, journals, funders, schools, and universities are struggling with a surge of potentially AI-generated submissions from ostensibly human authors, who may not have exerci…
ML Researchers Support Openness in Peer Review But Are Concerned About Resubmission Bias
Vishisht Rao, Justin Payan, Andrew McCallum +1
Peer-review venues have increasingly adopted open reviewing policies that publicly release anonymized reviews and permit public commenting. Venues have adopted a variety of policie…
FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers
Sarina Xi, Vishisht Rao, Justin Payan +1
The identification and localization of errors is a core task in peer review, yet the exponential growth of scientific output has made it increasingly difficult for human reviewers…
InstructExcel: A Benchmark for Natural Language Instruction in Excel
Justin Payan, Swaroop Mishra, Mukul Singh +7
With the evolution of Large Language Models (LLMs) we can solve increasingly more complex NLP tasks across various domains, including spreadsheets. This work investigates whether L…