5 papers
From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python
Muntasir Adnan, Manile Srun, Carlos C. N. Kuhn
The original ALPHA benchmark introduced a taxonomy-aware penalty for evaluating CWE-level vulnerability prediction in Python and proposed that the penalty could theoretically also…
A Systematic Investigation of Document Chunking Strategies and Embedding Sensitivity
Muhammad Arslan Shaukat, Muntasir Adnan, Carlos C. N. Kuhn
We present the first large-scale, cross-domain evaluation of document chunking strategies for dense retrieval, addressing a critical but underexplored aspect of retrieval-augmented…
Adaptive Hierarchical Evaluation of LLMs and SAST tools for CWE Prediction in Python
Muntasir Adnan, Carlos C. N. Kuhn
Large Language Models have become integral to software development, yet they frequently generate vulnerable code. Existing code vulnerability detection benchmarks employ binary cla…
The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMs
Muntasir Adnan, Carlos C. N. Kuhn
The effectiveness of AI debugging follows a predictable exponential decay pattern; most models lose 60-80% of their debugging capability within just 2-3 attempts, despite iterative…
Large Language Model Guided Self-Debugging Code Generation
Muntasir Adnan, Zhiwei Xu, Carlos C. N. Kuhn
Automated code generation is gaining significant importance in intelligent computer programming and system deployment. However, current approaches often face challenges in computat…