4 papers
Prism: Dynamic and Flexible Benchmarking of LLMs Code Generation with Monte Carlo Tree Search
Vahid Majdinasab, Amin Nikanjam, Foutse Khomh
The rapid advancement of Large Language Models (LLMs) has outpaced traditional evaluation methods. Static benchmarks fail to capture the depth and breadth of LLM capabilities and e…
DeepCodeProbe: Towards Understanding What Models Trained on Code Learn
Vahid Majdinasab, Amin Nikanjam, Foutse Khomh
Machine learning models trained on code and related artifacts offer valuable support for software maintenance but suffer from interpretability issues due to their complex internal…
Trained Without My Consent: Detecting Code Inclusion In Language Models Trained on Code
Vahid Majdinasab, Amin Nikanjam, Foutse Khomh
Code auditing ensures that the developed code adheres to standards, regulations, and copyright protection by verifying that it does not contain code from protected sources. The rec…
Assessing the Security of GitHub Copilot Generated Code -- A Targeted Replication Study
Vahid Majdinasab, Michael Joshua Bishop, Shawn Rasheed +3
AI-powered code generation models have been developing rapidly, allowing developers to expedite code generation and thus improve their productivity. These models are trained on lar…