7 papers
Sumi: Open Uniform Diffusion Language Model from Scratch
Mengyu Ye, Keito Kudo, Wataru Ikeda +3
Diffusion models have become a promising alternative to autoregressive models. Among these, uniform diffusion language models (UDLMs) permit any token to be updated at any step, in…
Reconsidering Positional Supervision in Masked Diffusion Language Model Training
Mengyu Ye, Keito Kudo, Ryosuke Takahashi +1
Masked diffusion language models (MDLMs) generate text by unmasking tokens in parallel and have recently emerged as alternatives to autoregressive language models. They can be view…
SLVMEval: Synthetic Meta Evaluation Benchmark for Text-to-Long Video Generation
Ryosuke Matsuda, Keito Kudo, Haruto Yoshida +2
This paper proposes the synthetic long-video meta-evaluation (SLVMEval), a benchmark for meta-evaluating text-to-video (T2V) evaluation systems. The proposed SLVMEval benchmark foc…
LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi +5
This study investigates the internal information flow of large language models (LLMs) while performing chain-of-thought (CoT) style reasoning. Specifically, with a particular inter…
Nodes Are Early, Edges Are Late: Probing Diagram Representations in Large Vision-Language Models
Haruto Yoshida, Keito Kudo, Yoichi Aoki +4
Large vision-language models (LVLMs) demonstrate strong performance on diagram understanding benchmarks, yet they still struggle with understanding relationships between elements,…
Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference
Go Kamoda, Benjamin Heinzerling, Tatsuro Inaba +3
According to the stages-of-inference hypothesis, early layers of language models map their subword-tokenized input, which does not necessarily correspond to a linguistically meanin…