Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representations
arXiv:2307.13776 · doi:10.18653/v1/2022.naacl-main.176
Abstract
In this paper, we advocate for using large pre-trained monolingual language models in cross lingual zero-shot word sense disambiguation (WSD) coupled with a contextualized mapping mechanism. We also report rigorous experiments that illustrate the effectiveness of employing sparse contextualized word representations obtained via a dictionary learning procedure. Our experimental results demonstrate that the above modifications yield a significant improvement of nearly 6.5 points of increase in the average F-score (from 62.0 to 68.5) over a collection of 17 typologically diverse set of target languages. We release our source code for replicating our experiments at https://github.com/begab/sparsity_makes_sense.
arXiv commentPresented at NAACL2022
References in corpus (6)
- Exploiting Similarities among Languages for Machine Translation
- Spanish Pre-trained BERT Model and Evaluation Data
- BERTje: A Dutch BERT Model
- Visualizing and Measuring the Geometry of BERT
- LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond
- word2word: A Collection of Bilingual Lexicons for 3,564 Language Pairs