1 citations · 1 across the 5 of their papers we have counts for
6 papers
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning
Mohammad Akbar-Tajari, Sara Rajaee, Mohammad Taher Pilehvar
Parameter-efficient fine-tuning approaches have recently garnered a lot of attention. Having considerably lower number of trainable weights, these methods can bring about scalabili…
Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference
Sara Rajaee, Yadollah Yaghoobzadeh, Mohammad Taher Pilehvar
It has been shown that NLI models are usually biased with respect to the word-overlap between premise and hypothesis; they take this feature as a primary cue for predicting the ent…
On the Importance of Data Size in Probing Fine-tuned Models
Houman Mehrafarin, Sara Rajaee, Mohammad Taher Pilehvar
Several studies have investigated the reasons behind the effectiveness of fine-tuning, usually through the lens of probing. However, these studies often neglect the role of the siz…
An Isotropy Analysis in the Multilingual BERT Embedding Space
Sara Rajaee, Mohammad Taher Pilehvar
Several studies have explored various advantages of multilingual pre-trained models (such as multilingual BERT) in capturing shared linguistic knowledge. However, less attention ha…
How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy
Sara Rajaee, Mohammad Taher Pilehvar
It is widely accepted that fine-tuning pre-trained language models usually brings about performance improvements in downstream tasks. However, there are limited studies on the reas…
A Cluster-based Approach for Improving Isotropy in Contextual Embedding Space
Sara Rajaee, Mohammad Taher Pilehvar
The representation degeneration problem in Contextual Word Representations (CWRs) hurts the expressiveness of the embedding space by forming an anisotropic cone where even unrelate…