3 citations · 3 across the 7 of their papers we have counts for
7 papers
The Poisoned Chalice of LLM Evaluation Report
Jonathan Katzy, Ali Al-Kaswan, Razvan Mihai Popescu +1
Large language models are increasingly used to evaluate and support software engineering tasks, yet the validity of these evaluations is often undermined by uncertainty about wheth…
Reliable and Developer-Aligned Evaluation of Agents for Software Engineering
Razvan Mihai Popescu
Large language models are rapidly moving towards closing the development cycle, transitioning from simple assistive companions to autonomous contributors deeply embedded into colla…
Automated Attention Pattern Discovery at Scale in Large Language Models
Jonathan Katzy, Razvan-Mihai Popescu, Erik Mekkes +2
Large language models have found success by scaling up capabilities to work in general settings. The same can unfortunately not be said for interpretability methods. The current tr…
Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time
Razvan Mihai Popescu, David Gros, Andrei Botocan +3
The rise of large language models for code has reshaped software development. Autonomous coding agents, able to create branches, open pull requests, and perform code reviews, now a…
The Heap: A Contamination-Free Multilingual Code Dataset for Evaluating Large Language Models
Jonathan Katzy, Razvan Mihai Popescu, Arie van Deursen +1
The recent rise in the popularity of large language models has spurred the development of extensive code datasets needed to train them. This has left limited code available for col…
An Exploratory Investigation into Code License Infringements in Large Language Model Training Datasets
Jonathan Katzy, Răzvan-Mihai Popescu, Arie van Deursen +1
Does the training of large language models potentially infringe upon code licenses? Furthermore, are there any datasets available that can be safely used for training these models…
Language Models for Code Completion: A Practical Evaluation
Maliheh Izadi, Jonathan Katzy, Tim van Dam +3
Transformer-based language models for automatic code completion have shown great promise so far, yet the evaluation of these models rarely uses real data. This study provides both…