BenchBench-Protocol: Evaluating Real-World Wet-Lab Protocol Reasoning and Modification

Aditya Sivakumar, Ashu Singhal, Nicholas Larus-Stone +1

We introduce BenchBench-Protocol, a benchmark for large language models of 149 protocol-modification tasks recovered from modifications that scientists made to published protocols…