5 papers
CASTLE: Benchmarking Dataset for Static Code Analyzers and LLMs towards CWE Detection
Richard A. Dubniczky, Krisztofer Zoltán Horvát, Tamás Bisztray +3
Identifying vulnerabilities in source code is crucial, especially in critical software components. Existing methods such as static analysis, dynamic analysis, formal verification,…
Vulnerability Detection: From Formal Verification to Large Language Models and Hybrid Approaches: A Comprehensive Overview
Norbert Tihanyi, Tamas Bisztray, Mohamed Amine Ferrag +4
Software testing and verification are critical for ensuring the reliability and security of modern software systems. Traditionally, formal verification techniques, such as model ch…
SecureFalcon: Are We There Yet in Automated Software Vulnerability Detection with LLMs?
Mohamed Amine Ferrag, Ammar Battah, Norbert Tihanyi +8
Software vulnerabilities can cause numerous problems, including crashes, data loss, and security breaches. These issues greatly compromise quality and can negatively impact the mar…
How secure is AI-generated Code: A Large-Scale Comparison of Large Language Models
Norbert Tihanyi, Tamas Bisztray, Mohamed Amine Ferrag +2
This study compares state-of-the-art Large Language Models (LLMs) on their tendency to generate vulnerabilities when writing C programs using a neutral zero-shot prompt. Tihanyi et…
Dynamic Intelligence Assessment: Benchmarking LLMs on the Road to AGI with a Focus on Model Confidence
Norbert Tihanyi, Tamas Bisztray, Richard A. Dubniczky +11
As machine intelligence evolves, the need to test and compare the problem-solving abilities of different AI models grows. However, current benchmarks are often simplistic, allowing…