From the 1 of 3 linked papers with an AI index.
3 papers
Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code
Rodrigo Pato Nogueira, Marco Vieira, João R. Campos
Large Language Models (LLMs) are being widely used for coding, with reports indicating that AI now generates an increasing share of production code. Studies show that LLMs can sign…
PROBE: Benchmarking Code Generation in Large Language Models
Rodrigo Pato Nogueira, Marco Vieira, João R. Campos
The paper presents PROBE, an extensible benchmark framework that evaluates large language models on code generation across functional correctness, solution proximity, and code qual…
A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code
Rodrigo Pato Nogueira, Karthik Pattabiraman, Marco Vieira +1
N-Version Programming (NVP) improves software reliability by executing multiple independent implementations and combining outputs, but its adoption is limited by high cost and the…