8 papers
AICrypto: Evaluating Cryptography Capabilities of Large Language Models
Yu Wang, Yijian Liu, Liheng Ji +11
We build \textbf{AICrypto}, a comprehensive benchmark designed to evaluate the cryptography capabilities of large language models (LLMs). The benchmark comprises 135 multiple-choic…
VirtualCrime: Evaluating the Criminal Potential and Agentic Behaviors of Large Language Models via Sandbox Simulation
Yilin Tang, Yu Wang, Lanlan Qiu +4
Large language models (LLMs) have shown strong capabilities in multi-step decision-making, and are increasingly integrated into real-world agentic applications. While existing safe…
Can Large Language Models Reinvent Foundational Algorithms?
Jian Zhao, Haoren Luo, Yu Wang +3
LLMs have shown strong potential to advance scientific discovery. Whether they possess the capacity for foundational innovation, however, remains an open question. In this work, we…
CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering
Baicheng Chen, Yu Wang, Ziheng Zhou +4
Reverse engineering (RE) is central to software security, particularly for cryptographic programs that handle sensitive data and are highly prone to vulnerabilities. It supports cr…
Pay for The Second-Best Service: A Game-Theoretic Approach Against Dishonest LLM Providers
Yuhan Cao, Yu Wang, Sitong Liu +3
The widespread adoption of Large Language Models (LLMs) through Application Programming Interfaces (APIs) induces a critical vulnerability: the potential for dishonest manipulation…
Can LLMs Generate Reliable Test Case Generators? A Study on Competition-Level Programming Problems
Yuhan Cao, Zian Chen, Kun Quan +17
Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation, capable of tackling complex tasks during inference. However, the extent to which LLMs can…