6 papers
AMNESIA: A Large Scale Medical Unlearning Benchmark Suite with Disease-Informed Analysis
Saeedeh Davoudi, Reihaneh Iranmanesh, Ophir Frieder +1
Medical knowledge is continuously evolving. This creates a need to update or selectively forget information encoded in already-trained medical LLMs. Machine unlearning aims to remo…
TARAZ: Persian Short-Answer Question Benchmark for Cultural Evaluation of Language Models
Reihaneh Iranmanesh, Saeedeh Davoudi, Pasha Abrishamchian +2
This paper presents a comprehensive evaluation framework for assessing the cultural competence of large language models (LLMs) in Persian. Existing Persian cultural benchmarks rely…
Generating Text from Uniform Meaning Representation
Emma Markle, Reihaneh Iranmanesh, Shira Wein
Uniform Meaning Representation (UMR) is a recently developed graph-based semantic representation, which expands on Abstract Meaning Representation (AMR) in a number of ways, in par…
The Structural Safety Generalization Problem
Julius Broomfield, Tom Gibbs, Ethan Kosak-Hine +7
LLM jailbreaks are a widespread safety challenge. Given this problem has not yet been tractable, we suggest targeting a key failure mechanism: the failure of safety to generalize a…
Shared Autonomy for Proximal Teaching
Megha Srivastava, Reihaneh Iranmanesh, Yuchen Cui +6
Motor skill learning often requires experienced professionals who can provide personalized instruction. Unfortunately, the availability of high-quality training can be limited for…
Emerging Vulnerabilities in Frontier Models: Multi-Turn Jailbreak Attacks
Tom Gibbs, Ethan Kosak-Hine, George Ingebretsen +6
Large language models (LLMs) are improving at an exceptional rate. However, these models are still susceptible to jailbreak attacks, which are becoming increasingly dangerous as mo…