13 citations · 40 across the 14 of their papers we have counts for
14 papers
Variable-length Neural Interlingua Representations for Zero-shot Neural Machine Translation
Zhuoyuan Mao, Haiyue Song, Raj Dabre +2
The language-independency of encoded representations within multilingual neural machine translation (MNMT) models is crucial for their generalization ability on zero-shot translati…
Exploring the Impact of Layer Normalization for Zero-shot Neural Machine Translation
Zhuoyuan Mao, Raj Dabre, Qianying Liu +3
This paper studies the impact of layer normalization (LayerNorm) on zero-shot translation (ZST). Recent efforts for ZST often utilize the Transformer architecture as the backbone,…
GPT-RE: In-context Learning for Relation Extraction using Large Language Models
Zhen Wan, Fei Cheng, Zhuoyuan Mao +4
In spite of the potential for ground-breaking achievements offered by large language models (LLMs) (e.g., GPT-3), they still lag significantly behind fully-supervised baselines (e.…
LEALLA: Learning Lightweight Language-agnostic Sentence Embeddings with Knowledge Distillation
Zhuoyuan Mao, Tetsuji Nakagawa
Large-scale language-agnostic sentence embedding models such as LaBSE (Feng et al., 2022) obtain state-of-the-art performance for parallel sentence alignment. However, these large-…
Textual Enhanced Contrastive Learning for Solving Math Word Problems
Yibin Shen, Qianying Liu, Zhuoyuan Mao +2
Solving math word problems is the task that analyses the relation of quantities and requires an accurate understanding of contextual natural language information. Recent studies sh…
Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction
Zhen Wan, Qianying Liu, Zhuoyuan Mao +3
Relation extraction (RE) has achieved remarkable progress with the help of pre-trained language models. However, existing RE models are usually incapable of handling two situations…
Seeking Diverse Reasoning Logic: Controlled Equation Expression Generation for Solving Math Word Problems
Yibin Shen, Qianying Liu, Zhuoyuan Mao +3
To solve Math Word Problems, human students leverage diverse reasoning logic that reaches different possible equation solutions. However, the mainstream sequence-to-sequence approa…
EMS: Efficient and Effective Massively Multilingual Sentence Embedding Learning
Zhuoyuan Mao, Chenhui Chu, Sadao Kurohashi
Massively multilingual sentence representation models, e.g., LASER, SBERT-distill, and LaBSE, help significantly improve cross-lingual downstream tasks. However, the use of a large…
Relation Extraction with Weighted Contrastive Pre-training on Distant Supervision
Zhen Wan, Fei Cheng, Qianying Liu +3
Contrastive pre-training on distant supervision has shown remarkable effectiveness in improving supervised relation extraction tasks. However, the existing methods ignore the intri…
When do Contrastive Word Alignments Improve Many-to-many Neural Machine Translation?
Zhuoyuan Mao, Chenhui Chu, Raj Dabre +3
Word alignment has proven to benefit many-to-many neural machine translation (NMT). However, high-quality ground-truth bilingual dictionaries were used for pre-editing in previous…
Linguistically-driven Multi-task Pre-training for Low-resource Neural Machine Translation
Zhuoyuan Mao, Chenhui Chu, Sadao Kurohashi
In the present study, we propose novel sequence-to-sequence pre-training objectives for low-resource machine translation (NMT): Japanese-specific sequence to sequence (JASS) for la…
Lightweight Cross-Lingual Sentence Representation Learning
Zhuoyuan Mao, Prakhar Gupta, Pei Wang +3
Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER (Artetxe and Schwenk, 2019b) lead to significant improvement in performance on d…
JASS: Japanese-specific Sequence to Sequence Pre-training for Neural Machine Translation
Zhuoyuan Mao, Fabien Cromieres, Raj Dabre +2
Neural machine translation (NMT) needs large parallel corpora for state-of-the-art translation quality. Low-resource NMT is typically addressed by transfer learning which leverages…
Pre-training via Leveraging Assisting Languages and Data Selection for Neural Machine Translation
Haiyue Song, Raj Dabre, Zhuoyuan Mao +3
Sequence-to-sequence (S2S) pre-training using large monolingual data is known to improve performance for various S2S NLP tasks in low-resource settings. However, large monolingual…