6 papers
AgentAbstain: Do LLM Agents Know When Not to Act?
Xun Liu, Yi Evie Zhang, Vira Kasprova +5
Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents…
Layer-Targeted Multilingual Knowledge Erasure in Large Language Models
Taoran Li, Varun Chandrasekaran, Zhiyuan Yu
Recent work has demonstrated that machine unlearning in Large Language Models (LLMs) fails to generalize across languages: knowledge erased in one language frequently remains acces…
AMUN: Adversarial Machine UNlearning
Ali Ebrahimpour-Boroojeny, Hari Sundaram, Varun Chandrasekaran
Machine unlearning, where users can request the deletion of a forget dataset, is becoming increasingly important because of numerous privacy regulations. Initial works on ``exact''…
BenchAgents: Multi-Agent Systems for Structured Benchmark Creation
Natasha Butt, Varun Chandrasekaran, Neel Joshi +2
Evaluation insights are limited by the availability of high-quality benchmarks. As models evolve, there is a need to create benchmarks that can measure progress on new and complex…
MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation
Siddharth Joshi, Besmira Nushi, Vidhisha Balachandran +4
Vision-language models (VLMs) are highly effective but often underperform on specialized tasks; for example, Llava-1.5 struggles with chart and diagram understanding due to scarce…
The Efficacy of Transfer-based No-box Attacks on Image Watermarking: A Pragmatic Analysis
Qilong Wu, Varun Chandrasekaran
Watermarking approaches are widely used to identify if images being circulated are authentic or AI-generated. Determining the robustness of image watermarking methods in the ``no-b…