13 citations · 34 across the 13 of their papers we have counts for
19 papers · 1 filter
Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embedding
Kaiyan Zhao, Qiyu Wu, Xin-Qiang Cai +1
Learning multi-lingual sentence embeddings is a fundamental task in natural language processing. Recent trends in learning both mono-lingual and multi-lingual sentence embeddings a…
WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Prediction
Qiyu Wu, Masaaki Nagata, Yoshimasa Tsuruoka
Most existing word alignment methods rely on manual alignment datasets or parallel corpora, which limits their usefulness. Here, to mitigate the dependence on manual data, we broad…
EASE: Entity-Aware Contrastive Learning of Sentence Embedding
Sosuke Nishikawa, Ryokan Ri, Ikuya Yamada +2
We present EASE, a novel method for learning sentence embeddings via contrastive learning between sentences and their related entities. The advantage of using entity supervision is…
Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models
Ryokan Ri, Yoshimasa Tsuruoka
We investigate what kind of structural knowledge learned in neural network encoders is transferable to processing natural language. We design artificial languages with structural p…
mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models
Ryokan Ri, Ikuya Yamada, Yoshimasa Tsuruoka
Recent studies have shown that multilingual pretrained language models can be effectively improved with cross-lingual alignment information from Wikipedia entities. However, existi…
A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification
Sosuke Nishikawa, Ikuya Yamada, Yoshimasa Tsuruoka +1
We present a multilingual bag-of-entities model that effectively boosts the performance of zero-shot cross-lingual text classification by extending a multilingual pre-trained langu…