3 papers
cs.CL2026
Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms
Yuto Nishida, Naoki Shikoda, Yosuke Kishinami +4
Understanding what kinds of factual knowledge large language models (LLMs) memorize is essential for evaluating their reliability and limitations. Entity-based QA is a common frame…
cs.CL2025
Instability in Downstream Task Performance During LLM Pretraining
Yuto Nishida, Masaru Isonuma, Yusuke Oda
When training large language models (LLMs), it is common practice to track downstream task performance throughout the training process and select the checkpoint with the highest va…
cs.CL2025
Long-Tail Crisis in Nearest Neighbor Language Models
Yuto Nishida, Makoto Morishita, Hiroyuki Deguchi +2
The -nearest-neighbor language model (NN-LM), one of the retrieval-augmented language models, improves the perplexity for given text by directly accessing a large datastore b…