9 papers
Searching for Privacy Risks in LLM Agents via Simulation
Yanzhe Zhang, Diyi Yang
The widespread deployment of LLM-based agents is likely to introduce a critical privacy threat: malicious agents that proactively engage others in multi-turn interactions to extrac…
Contextualized Privacy Defense for LLM Agents
Yule Wen, Yanzhe Zhang, Jianxun Lian +3
LLM agents increasingly act on users' personal information, yet existing privacy defenses remain limited in both design and adaptability. Most prior approaches rely on static or pa…
Towards Execution-Grounded Automated AI Research
Chenglei Si, Zitong Yang, Yejin Choi +3
Automated AI research holds great potential to accelerate scientific discovery. However, current LLMs often generate plausible-looking but ineffective ideas. Execution grounding ma…
AutoMetrics: Approximate Human Judgements with Automatically Generated Evaluators
Michael J. Ryan, Yanzhe Zhang, Amol Salunkhe +3
Evaluating user-facing AI applications remains a central challenge, especially in open-ended domains such as travel planning, clinical note generation, or dialogue. The gold standa…
The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
Chenglei Si, Tatsunori Hashimoto, Diyi Yang
Large Language Models (LLMs) have shown promise in accelerating the scientific research pipeline. A key capability for this process is the ability to generate novel research ideas,…
Attacking Vision-Language Computer Agents via Pop-ups
Yanzhe Zhang, Tao Yu, Diyi Yang
Autonomous agents powered by large vision and language models (VLM) have demonstrated significant potential in completing daily computer tasks, such as browsing the web to book tra…