Towards Faster k-Nearest-Neighbor Machine Translation
arXiv:2312.07419 · doi:10.54364/AAIML.2024.41111
Abstract
Recent works have proven the effectiveness of k-nearest-neighbor machine translation(a.k.a kNN-MT) approaches to produce remarkable improvement in cross-domain translations. However, these models suffer from heavy retrieve overhead on the entire datastore when decoding each token. We observe that during the decoding phase, about 67% to 84% of tokens are unvaried after searching over the corpus datastore, which means most of the tokens cause futile retrievals and introduce unnecessary computational costs by initiating k-nearest-neighbor searches. We consider this phenomenon is explainable in linguistics and propose a simple yet effective multi-layer perceptron (MLP) network to predict whether a token should be translated jointly by the neural machine translation model and probabilities produced by the kNN or just by the neural model. The results show that our method succeeds in reducing redundant retrieval operations and significantly reduces the overhead of kNN retrievals by up to 53% at the expense of a slight decline in translation quality. Moreover, our method could work together with all existing kNN-MT systems. This work has been accepted for publication in the jornal Advances in Artificial Intelligence and Machine Learning (ISSN: 2582-9793). The final published version can be found at DOI: https://dx.doi.org/10.54364/AAIML.2024.41111
7 pages
References in corpus (14)
- Adam: A Method for Stochastic Optimization
- Neural Machine Translation by Jointly Learning to Align and Translate
- Sequence to Sequence Learning with Neural Networks
- Categorical Reparameterization with Gumbel-Softmax
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- One-shot Learning with Memory-Augmented Neural Networks
- A Survey of Domain Adaptation for Neural Machine Translation
- Fast Nearest Neighbor Machine Translation
- Simple and Scalable Nearest Neighbor Machine Translation
- Efficient Cluster-Based k-Nearest-Neighbor Machine Translation
- kNN-BOX: A Unified Framework for Nearest Neighbor Generation
- Efficient Machine Translation Domain Adaptation
- Bridging the Domain Gaps in Context Representations for k-Nearest Neighbor Neural Machine Translation
- Towards Robust k-Nearest-Neighbor Machine Translation