1 citations · 1 across the 5 of their papers we have counts for
Showing 2021Show all
3 papers · 1 filter
cs.CL2021★ 1 cited
An Isotropy Analysis in the Multilingual BERT Embedding Space
Sara Rajaee, Mohammad Taher Pilehvar
Several studies have explored various advantages of multilingual pre-trained models (such as multilingual BERT) in capturing shared linguistic knowledge. However, less attention ha…
cs.CL2021
How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy
Sara Rajaee, Mohammad Taher Pilehvar
It is widely accepted that fine-tuning pre-trained language models usually brings about performance improvements in downstream tasks. However, there are limited studies on the reas…
cs.CL2021
A Cluster-based Approach for Improving Isotropy in Contextual Embedding Space
Sara Rajaee, Mohammad Taher Pilehvar
The representation degeneration problem in Contextual Word Representations (CWRs) hurts the expressiveness of the embedding space by forming an anisotropic cone where even unrelate…