6 papers
LLM-jp-4 Technical Report
LLM-jp, :, Chaoran Liu +16
This paper presents LLM-jp-4, a family of Japanese sovereign large language models (LLMs) developed with an emphasis on transparency, reproducibility, and practical usability. LLM-…
How Well Can LLMs Simulate Real Learner Evaluations of Educational Feedback?
Momoka Furuhashi, Kouta Nakayama, Takashi Kodama +2
While recent studies have explored human behavior and preference simulation using large language models (LLMs), it remains unclear how well LLMs can simulate subjective evaluations…
Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining
Yuto Nishida, Hirokazu Kiyomaru, Yusuke Oda +6
Measuring training data influence consistently across language model pretraining is challenging. It is difficult to select downstream tasks or validation sets representative of a m…
Investigating Learner-Aware Design of LLM-Generated Educational Feedback
Momoka Furuhashi, Kouta Nakayama, Noboru Kawai +3
Although large language models (LLMs) show promise for generating educational feedback, it remains unclear how feedback should be designed (e.g., tone and coverage) to support answ…
Are Checklists Really Useful for Automatic Evaluation of Generative Tasks?
Momoka Furuhashi, Kouta Nakayama, Takashi Kodama +1
Automatic evaluation of generative tasks using large language models faces challenges due to ambiguous criteria. Although automatic checklist generation is a potentially promising…
AnswerCarefully: A Dataset for Improving the Safety of Japanese LLM Output
Hisami Suzuki, Satoru Katsumata, Takashi Kodama +3
In this paper we present AnswerCarefully, a dataset for promoting the safety and appropriateness of Japanese LLM outputs. The dataset consists of 1,800 pairs of questions and refer…