5 papers
What Survives the Next Model? Benchmarking LLM-Based Techniques Against Single-Prompts
Nahian Salsabil, Joy Saha, Simantika Bhattacharjee Dristi +4
The software engineering research community has enthusiastically embraced the integration of Large Language Models (LLMs) into complex techniques to solve a wide variety of tasks.…
SONAR: Task-Aware Code Summary Evaluation for LLM Consumers Without References
Simantika Bhattacharjee Dristi, Matthew B. Dwyer
Source code summaries have traditionally been evaluated from a human developer's perspective, with quality determined by how closely they resemble developer-written references and…
A Differential Fuzzing-Based Evaluation of Functional Equivalence in LLM-Generated Code Refactorings
Simantika Bhattacharjee Dristi, Matthew B. Dwyer
With the rapid adoption of large language models (LLMs) in automated code refactoring, assessing and ensuring functional equivalence between LLM-generated refactoring and the origi…
Analyzing and Mitigating Surface Bias in Code Evaluation Metrics
Simantika Bhattacharjee Dristi, Matthew B. Dwyer
With the increasing popularity of large language models (LLMs) and LLM-based agents, reliable and effective code evaluation metrics (CEMs) have become crucial for progress across s…
Generating Realistic, Diverse, and Fault-Revealing Inputs with Latent Space Interpolation for Testing Deep Neural Networks
Bin Duan, Matthew B. Dwyer, Guowei Yang
Deep Neural Networks (DNNs) have been widely employed across various domains, including safety-critical systems, necessitating comprehensive testing to ensure their reliability. Al…