12 papers
Scaling Law: Optimizing Multi-Epoch, Multi-Lingual, and Multi-Stage Training for Low-Resource Language Models
Kosuke Akimoto, Taiki Miyagawa, Masafumi Oyamada
In this paper, we study a fundamental design problem in pretraining Large Language Models (LLMs) for low-resource language regimes. Existing works adopt multi-epoch, multi-lingual,…
Learning Deliberately, Acting Intuitively: Unlocking Test-Time Reasoning in Multimodal LLMs
Yahan Yu, Yuyang Dong, Masafumi Oyamada
Reasoning is essential for large language models (LLMs), especially in complex tasks such as mathematical problem solving. However, multimodal reasoning still faces challenges in m…
Evaluating the Impact of Reviewer Guideline Design on LLM-Based Automated Peer Review
Haowen Li, Yoichi Ishibashi, Masafumi Oyamada
Peer review is an essential process in scientific research, yet the growing workload has made its automation increasingly necessary. In this study, we analyze how different types o…
Best-of- -- Asymptotic Performance of Test-Time LLM Ensembling
Junpei Komiyama, Daisuke Oba, Masafumi Oyamada
We study best-of- for large language models (LLMs) where the selection is based on majority voting. In particular, we analyze the limit , which we denote as \boinf…
SCAN: Semantic Document Layout Analysis for Textual and Visual Retrieval-Augmented Generation
Nobuhiro Ueda, Yuyang Dong, Krisztián Boros +3
With the increasing adoption of Large Language Models (LLMs) and Vision-Language Models (VLMs), rich document analysis technologies for applications like Retrieval-Augmented Genera…
Can a Crow Hatch a Falcon? Lineage Matters in Predicting Large Language Model Performance
Takuya Tamura, Taro Yano, Masafumi Enomoto +1
Accurately forecasting the performance of Large Language Models (LLMs) before extensive fine-tuning or merging can substantially reduce both computational expense and development t…