30 citations · 43 across the 11 of their papers we have counts for
11 papers
The Poisoned Chalice of LLM Evaluation Report
Jonathan Katzy, Ali Al-Kaswan, Razvan Mihai Popescu +1
Large language models are increasingly used to evaluate and support software engineering tasks, yet the validity of these evaluations is often undermined by uncertainty about wheth…
Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture the Flag Challenges
Ali Al-Kaswan, Maksim Plotnikov, Maxim Hájek +3
Large Language Model (LLM) agents are increasingly proposed for autonomous cybersecurity tasks, but their capabilities in realistic offensive settings remain poorly understood. We…
AST-PAC: AST-guided Membership Inference for Code
Roham Koohestani, Ali Al-Kaswan, Jonathan Katzy +1
Code Large Language Models are frequently trained on massive datasets containing restrictively licensed source code. This creates urgent data governance and copyright challenges. M…
Model See, Model Do? Exposure-Aware Evaluation of Bug-vs-Fix Preference in Code LLMs
Ali Al-Kaswan, Claudio Spiess, Prem Devanbu +2
Large language models are increasingly used for code generation and debugging, but their outputs can still contain bugs, that originate from training data. Distinguishing whether a…
Code Red! On the Harmfulness of Applying Off-the-shelf Large Language Models to Programming Tasks
Ali Al-Kaswan, Sebastian Deatc, Begüm Koç +2
Nowadays, developers increasingly rely on solutions powered by Large Language Models (LLM) to assist them with their coding tasks. This makes it crucial to align these tools with h…
How Much Do Code Language Models Remember? An Investigation on Data Extraction Attacks before and after Fine-tuning
Fabio Salerno, Ali Al-Kaswan, Maliheh Izadi
Code language models, while widely popular, are often trained on unsanitized source code gathered from across the Internet. Previous work revealed that pre-trained models can remem…
Traces of Memorisation in Large Language Models for Code
Ali Al-Kaswan, Maliheh Izadi, Arie van Deursen
Large language models have gained significant popularity because of their ability to generate human-like text and potential applications in various fields, such as Software Enginee…
STACC: Code Comment Classification using SentenceTransformers
Ali Al-Kaswan, Maliheh Izadi, Arie van Deursen
Code comments are a key resource for information about software artefacts. Depending on the use case, only some types of comments are useful. Thus, automatic approaches to classify…
The (ab)use of Open Source Code to Train Large Language Models
Ali Al-Kaswan, Maliheh Izadi
In recent years, Large Language Models (LLMs) have gained significant popularity due to their ability to generate human-like text and their potential applications in various fields…
Targeted Attack on GPT-Neo for the SATML Language Model Data Extraction Challenge
Ali Al-Kaswan, Maliheh Izadi, Arie van Deursen
Previous work has shown that Large Language Models are susceptible to so-called data extraction attacks. This allows an attacker to extract a sample that was contained in the train…
Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries
Ali Al-Kaswan, Toufique Ahmed, Maliheh Izadi +3
Reverse engineering binaries is required to understand and analyse programs for which the source code is unavailable. Decompilers can transform the largely unreadable binaries into…