Long-Tail Crisis in Nearest Neighbor Language Models

Yuto Nishida, Makoto Morishita, Hiroyuki Deguchi +2

The $k$-nearest-neighbor language model ($k$NN-LM), one of the retrieval-augmented language models, improves the perplexity for given text by directly accessing a large datastore b…