54 citations · 74 across the 15 of their papers we have counts for
5 papers · 1 filter
LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks
Saad Ullah, Mingji Han, Saurabh Pujar +3
Large Language Models (LLMs) have been suggested for use in automated vulnerability repair, but benchmarks showing they can consistently identify security-related bugs are lacking.…
Cross-lingual Transfer in Programming Languages: An Extensive Empirical Study
Razan Baltaji, Saurabh Pujar, Louis Mandel +3
Large language models (LLMs) have achieved state-of-the-art performance in various software engineering tasks, including error detection, clone detection, and code translation, pri…
Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain
Marcus J. Min, Yangruibo Ding, Luca Buratti +4
Code Large Language Models (Code LLMs) are being increasingly employed in real-life applications, so evaluating them is critical. While the conventional accuracy evaluates the perf…
CONCORD: Clone-aware Contrastive Learning for Source Code
Yangruibo Ding, Saikat Chakraborty, Luca Buratti +4
Deep Learning (DL) models to analyze source code have shown immense promise during the past few years. More recently, self-supervised pre-training has gained traction for learning…
Automated Code generation for Information Technology Tasks in YAML through Large Language Models
Saurabh Pujar, Luca Buratti, Xiaojie Guo +8
The recent improvement in code generation capabilities due to the use of large language models has mainly benefited general purpose programming languages. Domain specific languages…