10 citations · 33 across the 39 of their papers we have counts for
6 papers · 1 filter
An Isotropy Analysis in the Multilingual BERT Embedding Space
Sara Rajaee, Mohammad Taher Pilehvar
Several studies have explored various advantages of multilingual pre-trained models (such as multilingual BERT) in capturing shared linguistic knowledge. However, less attention ha…
Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations
Mohsen Fayyaz, Ehsan Aghazadeh, Ali Modarressi +2
Most of the recent works on probing representations have focused on BERT, with the presumption that the findings might be similar to the other models. In this work, we extend the p…
How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy
Sara Rajaee, Mohammad Taher Pilehvar
It is widely accepted that fine-tuning pre-trained language models usually brings about performance improvements in downstream tasks. However, there are limited studies on the reas…
Don't Discard All the Biased Instances: Investigating a Core Assumption in Dataset Bias Mitigation Techniques
Hossein Amirkhani, Mohammad Taher Pilehvar
Existing techniques for mitigating dataset bias often leverage a biased model to identify biased instances. The role of these biased instances is then reduced during the training o…
A Cluster-based Approach for Improving Isotropy in Contextual Embedding Space
Sara Rajaee, Mohammad Taher Pilehvar
The representation degeneration problem in Contextual Word Representations (CWRs) hurts the expressiveness of the embedding space by forming an anisotropic cone where even unrelate…
Exploring the Role of BERT Token Representations to Explain Sentence Probing Results
Hosein Mohebbi, Ali Modarressi, Mohammad Taher Pilehvar
Several studies have been carried out on revealing linguistic features captured by BERT. This is usually achieved by training a diagnostic classifier on the representations obtaine…