From the 2 of 5 linked papers with an AI index.
5 papers
Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code
Rodrigo Pato Nogueira, Marco Vieira, João R. Campos
Large Language Models (LLMs) are being widely used for coding, with reports indicating that AI now generates an increasing share of production code. Studies show that LLMs can sign…
PROBE: Benchmarking Code Generation in Large Language Models
Rodrigo Pato Nogueira, Marco Vieira, João R. Campos
The paper presents PROBE, an extensible benchmark framework that evaluates large language models on code generation across functional correctness, solution proximity, and code qual…
Leveraging Large Language Models for Trustworthiness Assessment of Web Applications
Oleksandr Yarotskyi, José D'Abruzzo Pereira, João R. Campos
The paper proposes an automated method that uses large language models to evaluate web applications' trustworthiness by checking their adherence to secure coding practices and comp…
Effectiveness of LLM-based Software Diversity for Reliability Improvement -- an Empirical Study
Gabriel Almeida, Ilir Gashi, Vladimir Stankovic +1
Software diversity has been extensively studied as a means of reducing the risk of common-mode failures. Classic work showed that the central issue is whether failures of diversely…
A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code
Rodrigo Pato Nogueira, Karthik Pattabiraman, Marco Vieira +1
N-Version Programming (NVP) improves software reliability by executing multiple independent implementations and combining outputs, but its adoption is limited by high cost and the…