13 citations · 16 across the 8 of their papers we have counts for
9 papers
CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilities
Tianneng Shi, Robin Rheem, Dongwei Jiang +13
AI has the potential to transform cybersecurity by enabling systems that can autonomously detect, analyze, and remediate software vulnerabilities. However, existing cybersecurity e…
Supplement Generation Training for Enhancing Agentic Task Performance
Young Min Cho, Daniele Bonadiman, Divya Bhargavi +8
Training large foundation models for agentic tasks is increasingly impractical due to the high computational costs, long iteration cycles, and rapid obsolescence as new models are…
Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback
Dongwei Jiang, Alvin Zhang, Andrew Wang +2
Recent studies have shown LLMs possess some ability to improve their responses when given external feedback. However, it remains unclear how effectively and thoroughly these models…
RATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoning
Dongwei Jiang, Guoxuan Wang, Yining Lu +5
The reasoning steps generated by LLMs might be incomplete, as they mimic logical leaps common in everyday communication found in their pre-training data: underlying rationales are…
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez +7
Chain-of-thought (CoT) via prompting is the de facto method for eliciting reasoning capabilities from large language models (LLMs). But for what kinds of tasks is this extra ``thin…
Benchmarking Language Model Creativity: A Case Study on Code Generation
Yining Lu, Dixuan Wang, Tianjian Li +4
As LLMs become increasingly prevalent, it is interesting to consider how ``creative'' these models can be. From cognitive science, creativity consists of at least two key character…
SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses
Dongwei Jiang, Jingyu Zhang, Orion Weller +3
Can LLMs consistently improve their previous outputs for better results? For this to be true, LLMs would need to be better at discriminating among previously-generated alternatives…
LeanReasoner: Boosting Complex Logical Reasoning with Lean
Dongwei Jiang, Marcio Fonseca, Shay B. Cohen
Large language models (LLMs) often struggle with complex logical reasoning due to logical inconsistencies and the inherent difficulty of such reasoning. We use Lean, a theorem prov…
Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic
Nathaniel Weir, Kate Sanders, Orion Weller +8
Recent language models enable new opportunities for structured reasoning with text, such as the construction of intuitive, proof-like textual entailment trees without relying on br…