4 papers
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
An Embarrassingly Simple Defense Against LLM Abliteration Attacks
Harethah Abu Shairah, Hasan Abed Al Kader Hammoud, Bernard Ghanem +1
Large language models (LLMs) are typically aligned to refuse harmful instructions through safety fine-tuning. A recent attack, termed abliteration, identifies and suppresses the si…
ALARB: An Arabic Legal Argument Reasoning Benchmark
Harethah Abu Shairah, Somayah AlHarbi, Abdulaziz AlHussein +5
We introduce ALARB, a dataset and suite of tasks designed to evaluate the reasoning capabilities of large language models (LLMs) within the Arabic legal domain. While existing Arab…
Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection
Harethah Abu Shairah, Hasan Abed Al Kader Hammoud, George Turkiyyah +1
Safety alignment in Large Language Models (LLMs) often involves mediating internal representations to refuse harmful requests. Recent research has demonstrated that these safety me…