4 papers
Scaling Sparse Fine-Tuning to Large Language Models
Alan Ansell, Ivan Vulić, Hannah Sterz +2
Large Language Models (LLMs) are difficult to fully fine-tune (e.g., with instructions or human feedback) due to their sheer number of parameters. A family of parameter-efficient s…
Are Large Language Models Temporally Grounded?
Yifu Qiu, Zheng Zhao, Yftah Ziser +3
Are Large language models (LLMs) temporally grounded? Since LLMs cannot perceive and interact with the environment, it is impossible to answer this question directly. Instead, we p…
Distilling Efficient Language-Specific Models for Cross-Lingual Transfer
Alan Ansell, Edoardo Maria Ponti, Anna Korhonen +1
Massively multilingual Transformers (MMTs), such as mBERT and XLM-R, are widely used for cross-lingual transfer learning. While these are pretrained to represent hundreds of langua…
Elastic Weight Removal for Faithful and Abstractive Dialogue Generation
Nico Daheim, Nouha Dziri, Mrinmaya Sachan +2
Ideally, dialogue systems should generate responses that are faithful to the knowledge contained in relevant documents. However, many models generate hallucinated responses instead…