5 citations · 6 across the 6 of their papers we have counts for
3 papers · 1 filter
Scaling Law: Optimizing Multi-Epoch, Multi-Lingual, and Multi-Stage Training for Low-Resource Language Models
Kosuke Akimoto, Taiki Miyagawa, Masafumi Oyamada
In this paper, we study a fundamental design problem in pretraining Large Language Models (LLMs) for low-resource language regimes. Existing works adopt multi-epoch, multi-lingual,…
LightPAL: Lightweight Passage Retrieval for Open Domain Multi-Document Summarization
Masafumi Enomoto, Kunihiro Takeoka, Kosuke Akimoto +2
Open-Domain Multi-Document Summarization (ODMDS) is the task of generating summaries from large document collections in response to user queries. This task is crucial for efficient…
Context Quality Matters in Training Fusion-in-Decoder for Extractive Open-Domain Question Answering
Kosuke Akimoto, Kunihiro Takeoka, Masafumi Oyamada
Retrieval-augmented generation models augment knowledge encoded in a language model by providing additional relevant external knowledge (context) during generation. Although it has…