Who Evaluates the Evaluators? On Automatic Metrics for Assessing AI-based Offensive Code Generators

Abstract

AI-based code generators are an emerging solution for automatically writing programs starting from descriptions in natural language, by using deep neural networks (Neural Machine Translation, NMT). In particular, code generators have been used for ethical hacking and offensive security testing by generating proof-of-concept attacks. Unfortunately, the evaluation of code generators still faces several issues. The current practice uses output similarity metrics, i.e., automatic metrics that compute the textual similarity of generated code with ground-truth references. However, it is not clear what metric to use, and which metric is most suitable for specific contexts. This work analyzes a large set of output similarity metrics on offensive code generators. We apply the metrics on two state-of-the-art NMT models using two datasets containing offensive assembly and Python code with their descriptions in the English language. We compare the estimates from the automatic metrics with human evaluation and provide practical insights into their strengths and limitations.

References in corpus (2)

• Evaluating Large Language Models Trained on Code
• Can We Generate Shellcodes via Natural Language? An Empirical Study

Cited by in corpus (6)

• Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks
• Automating the Correctness Assessment of AI-generated Code for Security Contexts
• AI Code Generators for Security: Friend or Foe?
• Assessing Evaluation Metrics for Neural Test Oracle Generation
• Poisoning Programs by Un-Repairing Code: Security Concerns of AI-generated Code
• Enhancing Robustness of AI Offensive Code Generators via Data Augmentation