6 papers
AgentReview: Exploring Peer Review Dynamics with LLM Agents
Yiqiao Jin, Qinlin Zhao, Yiyang Wang +4
Peer review is fundamental to the integrity and advancement of scientific publication. Traditional methods of peer review analyses often rely on exploration and statistics of exist…
Understanding and Mitigating Bias Inheritance in LLM-based Data Augmentation on Downstream Tasks
Miaomiao Li, Hao Chen, Yang Wang +5
Generating synthetic datasets via large language models (LLMs) has emerged as a promising approach to improve LLM performance. However, LLMs inherently reflect biases in their trai…
Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent Approach
Yuchen Wu, Edward Sun, Kaijie Zhu +4
Large language models (LLMs) typically generate identical or similar responses for all users given the same prompt, posing serious safety risks in high-stakes applications where us…
NPHardEval4V: Dynamic Evaluation of Large Vision-Language Models with Effects of Vision
Xiang Li, Wenyue Hua, Kaijie Zhu +8
Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities in multimodal understanding, yet their reasoning abilities remain underexplored. Existing benchmarks…
MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents
Kaijie Zhu, Xianjun Yang, Jindong Wang +2
Recent research has explored that LLM agents are vulnerable to indirect prompt injection (IPI) attacks, where malicious tasks embedded in tool-retrieved information can redirect th…
SciEvo: A 2 Million, 30-Year Cross-disciplinary Dataset for Temporal Scientometric Analysis
Yiqiao Jin, Yijia Xiao, Yiyang Wang +1
Understanding the creation, evolution, and dissemination of scientific knowledge is crucial for bridging diverse subject areas and addressing complex global challenges such as pand…