6 papers
Enhancing Long Document Long Form Summarisation with Self-Planning
Xiaotang Du, Rohit Saxena, Laura Perez-Beltrachini +2
We introduce a novel approach for long context summarisation, highlight-guided generation, that leverages sentence-level information as a content plan to improve the traceability a…
Finding Culture-Sensitive Neurons in Vision-Language Models
Xiutian Zhao, Rochelle Choenni, Rohit Saxena +1
Despite their impressive performance, vision-language models (VLMs) still struggle on culturally situated inputs. To understand how VLMs process culturally grounded information, we…
Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models
Ameen Ali, Shahar Katz, Lior Wolf +1
Large language models (LLMs) often develop learned mechanisms specialized to specific datasets, such as reliance on domain-specific correlations, which yield high-confidence predic…
M-Wanda: Improving One-Shot Pruning for Multilingual LLMs
Rochelle Choenni, Ivan Titov
Multilingual LLM performance is often critically dependent on model size. With an eye on efficiency, this has led to a surge in interest in one-shot pruning methods that retain the…
Joint Localization and Activation Editing for Low-Resource Fine-Tuning
Wen Lai, Alexander Fraser, Ivan Titov
Parameter-efficient fine-tuning (PEFT) methods, such as LoRA, are commonly used to adapt LLMs. However, the effectiveness of standard PEFT methods is limited in low-resource scenar…
Mitigating Copy Bias in In-Context Learning through Neuron Pruning
Ameen Ali, Lior Wolf, Ivan Titov
Large language models (LLMs) have demonstrated impressive few-shot in-context learning (ICL) abilities. Still, we show that they are sometimes prone to a `copying bias', where they…