6 papers
DSWorld: A Data Science World Model for Efficient Autonomous Agents
Zherui Yang, Fan Liu, Hao Liu
Despite strong capabilities in data understanding and decision-making, autonomous data science agents still heavily rely on trial-and-error workflows that involve expensive computa…
RiskNet: A large-scale dataset of AI risk incidents from news with alignment and multi-dimensional annotations
Leihan Zhang, Wecheng Ye, Xianlong Ma +5
As artificial intelligence (AI) systems are increasingly deployed across socially consequential domains, reports of AI-related harms and failures have grown in frequency and divers…
MM-Agent: LLM as Agents for Real-world Mathematical Modeling Problem
Fan Liu, Zherui Yang, Cancheng Liu +3
Mathematical modeling is a cornerstone of scientific discovery and engineering practice, enabling the translation of real-world problems into formal systems across domains such as…
TP-RAG: Benchmarking Retrieval-Augmented Large Language Model Agents for Spatiotemporal-Aware Travel Planning
Hang Ni, Fan Liu, Xinyu Ma +5
Large language models (LLMs) have shown promise in automating travel planning, yet they often fall short in addressing nuanced spatiotemporal rationality. While existing benchmarks…
Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs
Zhao Xu, Fan Liu, Hao Liu
Although Large Language Models (LLMs) have demonstrated significant capabilities in executing complex tasks in a zero-shot manner, they are susceptible to jailbreak attacks and can…
JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Framework
Fan Liu, Yue Feng, Zhao Xu +4
Despite advancements in enhancing LLM safety against jailbreak attacks, evaluating LLM defenses remains a challenge, with current methods often lacking explainability and generaliz…