From the 1 of 4 linked papers with an AI index.
4 papers
Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality
Saima Afrin, Alessandro Midolo, Camilo Escobar-Velásquez +5
The paper introduces a curated multilingual benchmark to study how the natural language of prompts influences code generation quality of large language models, evaluating functiona…
AEGIS: From Clues to Verdicts -- Graph-Guided Deep Vulnerability Reasoning via Dialectics and Meta-Auditing
Sen Fang, Weiyuan Ding, Zhezhen Cao +2
Large Language Models (LLMs) are increasingly adopted for vulnerability detection, yet their reasoning remains fundamentally unsound. We identify a root cause shared by both major…
EVALOOOP: A Self-Consistency-Centered Framework for Assessing Large Language Model Robustness in Programming
Sen Fang, Weiyuan Ding, Mengshi Zhang +2
Evaluating the programming robustness of large language models (LLMs) is paramount for ensuring their reliability in AI-based software development. However, adversarial attacks exh…
Smaller = Weaker? Benchmarking Robustness of Quantized LLMs in Code Generation
Sen Fang, Weiyuan Ding, Antonio Mastropaolo +1
Quantization has emerged as a mainstream method for compressing Large Language Models (LLMs), reducing memory requirements and accelerating inference without architectural modifica…