SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework

Abstract

Smart contracts are essential for managing digital assets in blockchain networks, highlighting the need for effective security measures. This paper introduces SmartLLMSentry, a novel framework that leverages large language models (LLMs), specifically ChatGPT with in-context training, to advance smart contract vulnerability detection. Traditional rule-based frameworks have limitations in integrating new detection rules efficiently. In contrast, SmartLLMSentry utilizes LLMs to streamline this process. We created a specialized dataset of five randomly selected vulnerabilities for model training and evaluation. Our results show an exact match accuracy of 91.1% with sufficient data, although GPT-4 demonstrated reduced performance compared to GPT-3 in rule generation. This study illustrates that SmartLLMSentry significantly enhances the speed and accuracy of vulnerability detection through LLMdriven rule integration, offering a new approach to improving Blockchain security and addressing previously underexplored vulnerabilities in smart contracts.

References in corpus (10)

• Slither: A Static Analysis Framework For Smart Contracts
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities
• SmartEmbed: A Tool for Clone and Bug Detection in Smart Contracts through Structural Code Embedding
• WizardCoder: Empowering Code Large Language Models with Evol-Instruct
• Reentrancy Vulnerability Identification in Ethereum Smart Contracts
• A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair
• LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning
• WANA: Symbolic Execution of Wasm Bytecode for Cross-Platform Smart Contract Vulnerability Detection
• Active Example Selection for In-Context Learning