4 papers
Improving Argument Saliency Coverage in Small LLMs for Long Legal Opinion Summarization via Sequence-Level Distillation
Mohamed Elaraby, Ahmed Elhady, Diane Litman
We show that sequence-level distillation from a capable long-context teacher model is a simple, annotation-free, and data-efficient strategy for improving argument saliency coverag…
Cross-lingual Self-Consistency for Multilingual Reasoning with Language Models
Ahmed Elhady, Eneko Agirre, Mikel Artetxe
Despite expanding their multilingual coverage, the advanced reasoning capabilities of LLMs remain largely confined to a few high-resource languages like English. To address this, w…
Emergent Abilities of Large Language Models under Continued Pretraining for Language Adaptation
Ahmed Elhady, Eneko Agirre, Mikel Artetxe
Continued pretraining (CPT) is a popular approach to adapt existing large language models (LLMs) to new languages. When doing so, it is common practice to include a portion of Engl…
WiCkeD: A Simple Method to Make Multiple Choice Benchmarks More Challenging
Ahmed Elhady, Eneko Agirre, Mikel Artetxe
We introduce WiCkeD, a simple method to increase the complexity of existing multiple-choice benchmarks by randomly replacing a choice with "None of the above", a method often used…