6 papers
Finding Culture-Sensitive Neurons in Vision-Language Models
Xiutian Zhao, Rochelle Choenni, Rohit Saxena +1
Despite their impressive performance, vision-language models (VLMs) still struggle on culturally situated inputs. To understand how VLMs process culturally grounded information, we…
Enhancing Long Document Long Form Summarisation with Self-Planning
Xiaotang Du, Rohit Saxena, Laura Perez-Beltrachini +2
We introduce a novel approach for long context summarisation, highlight-guided generation, that leverages sentence-level information as a content plan to improve the traceability a…
Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models
Ameen Ali, Shahar Katz, Lior Wolf +1
Large language models (LLMs) often develop learned mechanisms specialized to specific datasets, such as reliance on domain-specific correlations, which yield high-confidence predic…
Joint Localization and Activation Editing for Low-Resource Fine-Tuning
Wen Lai, Alexander Fraser, Ivan Titov
Parameter-efficient fine-tuning (PEFT) methods, such as LoRA, are commonly used to adapt LLMs. However, the effectiveness of standard PEFT methods is limited in low-resource scenar…
M-Wanda: Improving One-Shot Pruning for Multilingual LLMs
Rochelle Choenni, Ivan Titov
Multilingual LLM performance is often critically dependent on model size. With an eye on efficiency, this has led to a surge in interest in one-shot pruning methods that retain the…
Mitigating Copy Bias in In-Context Learning through Neuron Pruning
Ameen Ali, Lior Wolf, Ivan Titov
Large language models (LLMs) have demonstrated impressive few-shot in-context learning (ICL) abilities. Still, we show that they are sometimes prone to a `copying bias', where they…