2 papers
cs.SE2024
CRQBench: A Benchmark of Code Reasoning Questions
Elizabeth Dinella, Satish Chandra, Petros Maniatis
Large Language Models have demonstrated exceptional proficiency on coding tasks, but it is challenging to precisely evaluate their code reasoning ability. Existing benchmarks are i…
cs.SE2022
Predictive Synthesis of API-Centric Code
Daye Nam, Baishakhi Ray, Seohyun Kim +2
Today's programmers, especially data science practitioners, make heavy use of data-processing libraries (APIs) such as PyTorch, Tensorflow, NumPy, Pandas, and the like. Program syn…