1 citations · 2 across the 7 of their papers we have counts for
8 papers
Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency
Lucas Bandarkar, Alan Ansell, Trevor Cohn
Modern LLMs continue to exhibit significant variance in behavior across languages, such as being able to recall factual information in some languages but not others. While typicall…
Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
Lucas Bandarkar, Alan Ansell, Trevor Cohn
In this work, we analyze shortcomings in cross-lingual knowledge transfer in large, modern reasoning LLMs. We demonstrate that the perceived gap in knowledge transfer is primarily…
Training Plug-n-Play Knowledge Modules with Deep Context Distillation
Lucas Caccia, Alan Ansell, Edoardo Ponti +2
Dynamically integrating new or rapidly evolving information after (Large) Language Model pre-training remains challenging, particularly in low-data scenarios or when dealing with p…
Scaling Sparse Fine-Tuning to Large Language Models
Alan Ansell, Ivan Vulić, Hannah Sterz +2
Large Language Models (LLMs) are difficult to fully fine-tune (e.g., with instructions or human feedback) due to their sheer number of parameters. A family of parameter-efficient s…
Cross-Lingual Transfer with Target Language-Ready Task Adapters
Marinela Parović, Alan Ansell, Ivan Vulić +1
Adapters have emerged as a modular and parameter-efficient approach to (zero-shot) cross-lingual transfer. The established MAD-X framework employs separate language and task adapte…
Distilling Efficient Language-Specific Models for Cross-Lingual Transfer
Alan Ansell, Edoardo Maria Ponti, Anna Korhonen +1
Massively multilingual Transformers (MMTs), such as mBERT and XLM-R, are widely used for cross-lingual transfer learning. While these are pretrained to represent hundreds of langua…
Composable Sparse Fine-Tuning for Cross-Lingual Transfer
Alan Ansell, Edoardo Maria Ponti, Anna Korhonen +1
Fine-tuning the entire set of parameters of a large pretrained model has become the mainstream approach for transfer learning. To increase its efficiency and prevent catastrophic f…
PolyLM: Learning about Polysemy through Language Modeling
Alan Ansell, Felipe Bravo-Marquez, Bernhard Pfahringer
To avoid the "meaning conflation deficiency" of word embeddings, a number of models have aimed to embed individual word senses. These methods at one time performed well on tasks su…