6 papers
Sumi: Open Uniform Diffusion Language Model from Scratch
Mengyu Ye, Keito Kudo, Wataru Ikeda +3
Diffusion models have become a promising alternative to autoregressive models. Among these, uniform diffusion language models (UDLMs) permit any token to be updated at any step, in…
Reconsidering Positional Supervision in Masked Diffusion Language Model Training
Mengyu Ye, Keito Kudo, Ryosuke Takahashi +1
Masked diffusion language models (MDLMs) generate text by unmasking tokens in parallel and have recently emerged as alternatives to autoregressive language models. They can be view…
Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages
Tarek Naous, Anagha Savit, Carlos Rafael Catalan +17
As Large Language Models (LLMs) develop stronger multilingual capabilities, their sensitivity to culturally diverse entities becomes increasingly important. Prior work by Naous et…
An Open and Reproducible Deep Research Agent for Long-Form Question Answering
Ikuya Yamada, Wataru Ikeda, Ko Yoshida +5
We present an open deep research system for long-form question answering, selected as a winning system in the text-to-text track of the MMU-RAG competition at NeurIPS 2025. The sys…
Transformer Key-Value Memories Are Nearly as Interpretable as Sparse Autoencoders
Mengyu Ye, Jun Suzuki, Tatsuro Inaba +1
Recent interpretability work on large language models (LLMs) has been increasingly dominated by a feature-discovery approach with the help of proxy modules. Then, the quality of fe…
Can Input Attributions Explain Inductive Reasoning in In-Context Learning?
Mengyu Ye, Tatsuki Kuribayashi, Goro Kobayashi +1
Interpreting the internal process of neural models has long been a challenge. This challenge remains relevant in the era of large language models (LLMs) and in-context learning (IC…