6 papers · 1 filter
Natural Fingerprints of Large Language Models
Teppei Suzuki, Ryokan Ri, Sho Takase
Recent studies have shown that the outputs from large language models (LLMs) can often reveal the identity of their source model. While this is a natural consequence of LLMs modeli…
Dynamic Injection of Entity Knowledge into Dense Retrievers
Ikuya Yamada, Ryokan Ri, Takeshi Kojima +2
Dense retrievers often struggle with queries involving less-frequent entities due to their limited entity knowledge. We propose the Knowledgeable Passage Retriever (KPR), a BERT-ba…
Self-Preference Bias in LLM-as-a-Judge
Koki Wataoka, Tsubasa Takahashi, Ryokan Ri
Automated evaluation leveraging large language models (LLMs), commonly referred to as LLM evaluators or LLM-as-a-judge, has been widely used in measuring the performance of dialogu…
Large Vocabulary Size Improves Large Language Models
Sho Takase, Ryokan Ri, Shun Kiyono +1
This paper empirically investigates the relationship between subword vocabulary size and the performance of large language models (LLMs) to provide insights on how to define the vo…
Self-Translate-Train: Enhancing Cross-Lingual Transfer of Large Language Models via Inherent Capability
Ryokan Ri, Shun Kiyono, Sho Takase
Zero-shot cross-lingual transfer by fine-tuning multilingual pretrained models shows promise for low-resource languages, but often suffers from misalignment of internal representat…
LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation
Ikuya Yamada, Ryokan Ri
Adapting English-based large language models (LLMs) to other languages has become increasingly popular due to the efficiency and potential of cross-lingual transfer. However, exist…