3 citations · 3 across the 11 of their papers we have counts for
9 papers · 1 filter
Sumi: Open Uniform Diffusion Language Model from Scratch
Mengyu Ye, Keito Kudo, Wataru Ikeda +3
Diffusion models have become a promising alternative to autoregressive models. Among these, uniform diffusion language models (UDLMs) permit any token to be updated at any step, in…
Nodes Are Early, Edges Are Late: Probing Diagram Representations in Large Vision-Language Models
Haruto Yoshida, Keito Kudo, Yoichi Aoki +4
Large vision-language models (LVLMs) demonstrate strong performance on diagram understanding benchmarks, yet they still struggle with understanding relationships between elements,…
Reconsidering Positional Supervision in Masked Diffusion Language Model Training
Mengyu Ye, Keito Kudo, Ryosuke Takahashi +1
Masked diffusion language models (MDLMs) generate text by unmasking tokens in parallel and have recently emerged as alternatives to autoregressive language models. They can be view…
Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference
Go Kamoda, Benjamin Heinzerling, Tatsuro Inaba +3
According to the stages-of-inference hypothesis, early layers of language models map their subword-tokenized input, which does not necessarily correspond to a linguistically meanin…
LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi +5
This study investigates the internal information flow of large language models (LLMs) while performing chain-of-thought (CoT) style reasoning. Specifically, with a particular inter…
First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model Reasoning
Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi +4
Multi-step reasoning instruction, such as chain-of-thought prompting, is widely adopted to explore better language models (LMs) performance. We report on the systematic strategy th…