54 citations · 74 across the 15 of their papers we have counts for
16 papers
One Recipe, Many Harnesses: What Self-Evolution Encodes Across Languages and Models
Siqi Yang, Qianlan Yang, Yu-Xiong Wang +2
Self-evolving harnesses are closed-loop systems in which an agent inspects its own rollouts and edits its prompts, tools, and memory. They reliably improve coding agents in evaluat…
How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
Ramtin Ehsani, Irene Manotas, Saurabh Pujar +2
Large Language Model (LLM)-based Automated Program Repair systems are advancing rapidly, yet their performance remains inconsistent. Even when provided with the same contextual inf…
Code Reasoning for Software Engineering Tasks: A Survey and A Call to Action
Saurabh Pujar, Ira Ceka, Irene Manotas +3
The rise of large language models (LLMs) has led to dramatic improvements across a wide range of natural language tasks. Their performance on certain tasks can be further enhanced…
Understanding Automated Program Repair Agents Through the Lens of Traceability: An Empirical Study
Ira Ceka, Hailie Mitchell, Saurabh Pujar +5
Automated Program Repair (APR) agents leverage large language models (LLMs) to autonomously diagnose and patch software bugs using planning, reasoning, and tools. Although these ag…
SeaView: Software Engineering Agent Visual Interface for Enhanced Workflow
Timothy Bula, Saurabh Pujar, Luca Buratti +2
Auto-regressive LLM-based software engineering (SWE) agents, henceforth SWE agents, have made tremendous progress (>60% on SWE-Bench Verified) on real-world coding challenges inclu…
Insights from the Usage of the Ansible Lightspeed Code Completion Service
Priyam Sahoo, Saurabh Pujar, Ganesh Nalawade +3
The availability of Large Language Models (LLMs) which can generate code, has made it possible to create tools that improve developer productivity. Integrated development environme…
LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks
Saad Ullah, Mingji Han, Saurabh Pujar +3
Large Language Models (LLMs) have been suggested for use in automated vulnerability repair, but benchmarks showing they can consistently identify security-related bugs are lacking.…
Cross-lingual Transfer in Programming Languages: An Extensive Empirical Study
Razan Baltaji, Saurabh Pujar, Louis Mandel +3
Large language models (LLMs) have achieved state-of-the-art performance in various software engineering tasks, including error detection, clone detection, and code translation, pri…
Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain
Marcus J. Min, Yangruibo Ding, Luca Buratti +4
Code Large Language Models (Code LLMs) are being increasingly employed in real-life applications, so evaluating them is critical. While the conventional accuracy evaluates the perf…
CONCORD: Clone-aware Contrastive Learning for Source Code
Yangruibo Ding, Saikat Chakraborty, Luca Buratti +4
Deep Learning (DL) models to analyze source code have shown immense promise during the past few years. More recently, self-supervised pre-training has gained traction for learning…
Automated Code generation for Information Technology Tasks in YAML through Large Language Models
Saurabh Pujar, Luca Buratti, Xiaojie Guo +8
The recent improvement in code generation capabilities due to the use of large language models has mainly benefited general purpose programming languages. Domain specific languages…
Towards Learning (Dis)-Similarity of Source Code from Program Contrasts
Yangruibo Ding, Luca Buratti, Saurabh Pujar +3
Understanding the functional (dis)-similarity of source code is significant for code modeling tasks such as software vulnerability and code clone detection. We present DISCO(DIS-si…
CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks
Ruchir Puri, David S. Kung, Geert Janssen +14
Over the last several decades, software has been woven into the fabric of every aspect of our society. As software development surges and code infrastructure of enterprise applicat…
D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis
Yunhui Zheng, Saurabh Pujar, Burn Lewis +6
Static analysis tools are widely used for vulnerability detection as they understand programs with complex behavior and millions of lines of code. Despite their popularity, static…
Exploring Software Naturalness through Neural Language Models
Luca Buratti, Saurabh Pujar, Mihaela Bornea +8
The Software Naturalness hypothesis argues that programming languages can be understood through the same techniques used in natural language processing. We explore this hypothesis…
The TechQA Dataset
Vittorio Castelli, Rishav Chakravarti, Saswati Dana +18
We introduce TechQA, a domain-adaptation question answering dataset for the technical support domain. The TechQA corpus highlights two real-world issues from the automated customer…