3 papers
cs.CL2026
Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms
Yuto Nishida, Naoki Shikoda, Yosuke Kishinami +4
Understanding what kinds of factual knowledge large language models (LLMs) memorize is essential for evaluating their reliability and limitations. Entity-based QA is a common frame…
cs.SE2026
TimeMachine-bench: A Benchmark for Evaluating Model Capabilities in Repository-Level Migration Tasks
Ryo Fujii, Makoto Morishita, Kazuki Yano +1
With the advancement of automated software engineering, research focus is increasingly shifting toward practical tasks reflecting the day-to-day work of software engineers. Among t…
cs.CL2020
PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents
Ryo Fujii, Masato Mita, Kaori Abe +4
Neural Machine Translation (NMT) has shown drastic improvement in its quality when translating clean input, such as text from the news domain. However, existing studies suggest tha…