LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Training

Abstract

Paraphrases are texts that convey the same meaning while using different words or sentence structures. It can be used as an automatic data augmentation tool for many Natural Language Processing tasks, especially when dealing with low-resource languages, where data shortage is a significant problem. To generate a paraphrase in multilingual settings, previous studies have leveraged the knowledge from the machine translation field, i.e., forming a paraphrase through zero-shot machine translation in the same language. Despite good performance on human evaluation, those methods still require parallel translation datasets, thus making them inapplicable to languages that do not have parallel corpora. To mitigate that problem, we proposed the first unsupervised multilingual paraphrasing model, LAMPAT ($\text{L}$ow-rank $\text{A}$daptation for $\text{M}$ultilingual $\text{P}$araphrasing using $\text{A}$dversarial $\text{T}$raining), by which monolingual dataset is sufficient enough to generate a human-like and diverse sentence. Throughout the experiments, we found out that our method not only works well for English but can generalize on unseen languages as well. Data and code are available at https://github.com/VinAIResearch/LAMPAT.

First two authors contribute equally. Accepted at AAAI 2024

References in corpus (14)

• Explaining and Harnessing Adversarial Examples
• LoRA: Low-Rank Adaptation of Large Language Models
• BERTScore: Evaluating Text Generation with BERT
• Cross-lingual Language Model Pretraining
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• Pre-training via Paraphrasing
• You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
• Towards Robustness Against Natural Language Word Substitutions
• Joint Copying and Restricted Generation for Paraphrase
• How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?
• Paraphrase Generation as Zero-Shot Multilingual Translation: Disentangling Semantic Similarity from Lexical and Syntactic Diversity
• PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
• Human-Paraphrased References Improve Neural Machine Translation
• Novelty Controlled Paraphrase Generation with Retrieval Augmented Conditional Prompt Tuning