Yusuke Oda

LLM-jp-4 Technical Report

LLM-jp, :, Chaoran Liu +16

This paper presents LLM-jp-4, a family of Japanese sovereign large language models (LLMs) developed with an emphasis on transparency, reproducibility, and practical usability. LLM-…

Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining

Yuto Nishida, Hirokazu Kiyomaru, Yusuke Oda +6

Measuring training data influence consistently across language model pretraining is challenging. It is difficult to select downstream tasks or validation sets representative of a m…

Detecting Sensitive Personal Information in Japanese Pre-Training Corpora for Large Language Models

Rei Minamoto, Yusuke Oda, Daisuke Kawahara

Sensitive personal information can appear in large-scale pre-training corpora for large language models (LLMs). Detecting and filtering such information is therefore essential to e…

HakushoBench: A Japanese Chart and Table VQA Benchmark from Governmental White Papers

Issa Sugiura, Shuhei Kurita, Yusuke Oda +1

Understanding chart and table images is essential for applying vision-language models (VLMs) to real-world document understanding. While English benchmarks have advanced rapidly, n…

ABMAMBA: Multimodal Large Language Model with Aligned Hierarchical Bidirectional Scan for Efficient Video Captioning

Daichi Yashima, Shuhei Kurita, Yusuke Oda +3

In this study, we focus on video captioning by fully open multimodal large language models (MLLMs). The comprehension of visual sequences is challenging because of their intricate…

JAMMEval: A Refined Collection of Japanese Benchmarks for Reliable VLM Evaluation

Issa Sugiura, Koki Maeda, Shuhei Kurita +3

Reliable evaluation is essential for the development of vision-language models (VLMs). However, Japanese VQA benchmarks have undergone far less iterative refinement than their Engl…

Jagle: Building a Large-Scale Japanese Multimodal Post-Training Dataset for Vision-Language Models

Issa Sugiura, Keito Sasagawa, Keisuke Nakao +8

Developing vision-language models (VLMs) that generalize across diverse tasks requires large-scale training datasets with diverse content. In English, such datasets are typically c…

cs.RO2026

HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching

Daichi Yashima, Koki Seno, Shuhei Kurita +2

Coarse-to-fine autoregressive modeling has recently shown strong promise for visuomotor policy learning, combining the inference efficiency of autoregressive methods with the globa…

ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws

Xuyang Cao, Qianying Liu, Chuan Xiao +7

In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, namely the \textit{language mixtu…

ReMoRa: Multimodal Large Language Model based on Refined Motion Representation for Long-Video Understanding

Daichi Yashima, Shuhei Kurita, Yusuke Oda +1

While multimodal large language models (MLLMs) have shown remarkable success across a wide range of tasks, long-form video understanding remains a significant challenge. In this st…

Instability in Downstream Task Performance During LLM Pretraining

Yuto Nishida, Masaru Isonuma, Yusuke Oda

When training large language models (LLMs), it is common practice to track downstream task performance throughout the training process and select the checkpoint with the highest va…

cs.CV2025

WAON: A Large-Scale Japanese Image-Text Dataset for Cultural Adaptation in Contrastive Vision-Language Models

Issa Sugiura, Shuhei Kurita, Yusuke Oda +3

Contrastive vision-language models have achieved remarkable progress through large-scale pretraining. Recent work has shown that removing English-only caption filters and pretraini…

Llama-Mimi: Exploring the Limits of Flattened Speech Language Modeling

Issa Sugiura, Shuhei Kurita, Yusuke Oda +1

Speech Language Models (SpeechLMs) model tokenized speech to capture both semantic and acoustic information. When neural audio codecs based on Residual Vector Quantization (RVQ) ar…

Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality

Yuto Harada, Yusuke Yamauchi, Yusuke Oda +3

Supervised fine-tuning (SFT) is a critical step in aligning large language models (LLMs) with human instructions and values, yet many aspects of SFT remain poorly understood. We tr…

llm-jp-modernbert: A ModernBERT Model Trained on a Large-Scale Japanese Corpus with Long Context Length

Issa Sugiura, Kouta Nakayama, Yusuke Oda

Encoder-only transformer models like BERT are widely adopted as a pre-trained backbone for tasks like sentence classification and retrieval. However, pretraining of encoder models…