4 papers
Scam2Prompt: A Scalable Framework for Auditing Malicious Scam Endpoints in Production LLMs
Zhiyang Chen, Tara Saba, Xun Deng +2
Large Language Models have become critical to modern software development, but their reliance on uncurated web-scale datasets for training introduces a significant security risk: t…
TypyBench: Evaluating LLM Type Inference for Untyped Python Repositories
Honghua Dong, Jiacheng Yang, Xun Deng +4
Type inference for dynamic languages like Python is a persistent challenge in software engineering. While large language models (LLMs) have shown promise in code understanding, the…
VerifyThisBench: Generating Code, Specifications, and Proofs All at Once
Xun Deng, Sicheng Zhong, Barış Bayazıt +3
Large language models (LLMs) have demonstrated remarkable progress in code generation, but many existing benchmarks are approaching saturation and offer little guarantee on the tru…
Assessing Code Generation with Intermediate Languages
Xun Deng, Sicheng Zhong, Honghua Dong +4
Intermediate step methodologies like chain of thoughts (COT) have demonstrated effectiveness in enhancing the performance of Large Language Models (LLMs) on code generation. This s…