7 papers
VHDLSuite: Unified Pipeline for LLM VHDL Generation with Data Synthesis and Evaluation
Yijun Shen, Minghao Shao, Yichen Zhao +4
Large Language Models (LLM) have shown impressive capabilities in Register Transfer Level (RTL) code generation, particularly for Verilog. However, evaluating their performance wit…
RAVEN: Retrieval-Augmented Vulnerability Exploration Network for Memory Corruption Analysis in User Code and Binary Programs
Parteek Jamwal, Minghao Shao, Boyuan Chen +15
Large Language Models (LLMs) have demonstrated remarkable capabilities across various cybersecurity tasks, including vulnerability classification, detection, and patching. However,…
GroundCount: Grounding Vision-Language Models with Object Detection for Mitigating Counting Hallucinations
Boyuan Chen, Minghao Shao, Siddharth Garg +2
Vision Language Models (VLMs) exhibit persistent hallucinations in counting tasks, with accuracy substantially lower than other visual reasoning tasks (excluding sentiment). This p…
Steerable Instruction Following Coding Data Synthesis with Actor-Parametric Schema Co-Evolution
Tinglin Huang, Bo Chen, Xiao Zhang +2
Interpreting and following human instructions is a critical capability of large language models (LLMs) in automatic programming. However, synthesizing large-scale instruction-paire…
MetaCipher: A Time-Persistent and Universal Multi-Agent Framework for Cipher-Based Jailbreak Attacks for LLMs
Boyuan Chen, Minghao Shao, Abdul Basit +2
As large language models (LLMs) grow more capable, they face growing vulnerability to sophisticated jailbreak attacks. While developers invest heavily in alignment finetuning and s…
NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security
Minghao Shao, Sofija Jancheska, Meet Udeshi +10
Large Language Models (LLMs) are being deployed across various domains today. However, their capacity to solve Capture the Flag (CTF) challenges in cybersecurity has not been thoro…