66 citations · 71 across the 18 of their papers we have counts for
4 papers · 2 filters
LLM-jp-4 Technical Report
LLM-jp, :, Chaoran Liu +16
This paper presents LLM-jp-4, a family of Japanese sovereign large language models (LLMs) developed with an emphasis on transparency, reproducibility, and practical usability. LLM-…
Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining
Yuto Nishida, Hirokazu Kiyomaru, Yusuke Oda +6
Measuring training data influence consistently across language model pretraining is challenging. It is difficult to select downstream tasks or validation sets representative of a m…
Detecting Sensitive Personal Information in Japanese Pre-Training Corpora for Large Language Models
Rei Minamoto, Yusuke Oda, Daisuke Kawahara
Sensitive personal information can appear in large-scale pre-training corpora for large language models (LLMs). Detecting and filtering such information is therefore essential to e…
ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws
Xuyang Cao, Qianying Liu, Chuan Xiao +7
In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, namely the \textit{language mixtu…