36 citations · 47 across the 8 of their papers we have counts for
5 papers · 1 filter
Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention
Negar Foroutan, Mohammadreza Banaei, Karl Aberer +1
In this work, we study whether multilingual language models (MultiLMs) can transfer logical reasoning abilities to other languages when they are fine-tuned for reasoning in a diffe…
Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models
Mohammadreza Banaei, Klaudia Bałazy, Artur Kasymov +3
Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-…
Discovering Language-neutral Sub-networks in Multilingual Language Models
Negar Foroutan, Mohammadreza Banaei, Remi Lebret +2
Multilingual pre-trained language models transfer remarkably well on cross-lingual downstream tasks. However, the extent to which they learn language-neutral representations (i.e.,…
Direction is what you need: Improving Word Embedding Compression in Large Language Models
Klaudia Bałazy, Mohammadreza Banaei, Rémi Lebret +2
The adoption of Transformer-based models in natural language processing (NLP) has led to great success using a massive number of parameters. However, due to deployment constraints…
Spoken dialect identification in Twitter using a multi-filter architecture
Mohammadreza Banaei, Rémi Lebret, Karl Aberer
This paper presents our approach for SwissText & KONVENS 2020 shared task 2, which is a multi-stage neural model for Swiss German (GSW) identification on Twitter. Our model outputs…