6 papers · 1 filter
Beyond Perplexity: UTF-8 Validity in Byte-aware Language Models
Sangwhan Moon, Daisuke Oba, Youmi Ma +2
Byte-level tokenization enables language models to handle any Unicode input, but models can generate invalid UTF-8 sequences when encountering rare or unseen characters. We investi…
Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs
Sora Miyamoto, Daisuke Oba, Naoaki Okazaki
Tree-search decoding is an effective form of test-time scaling for large language models (LLMs), but real-world deployment often imposes a fixed per-query token budget that varies…
Drifting Objectives for Refining Discrete Diffusion Language Models
Daisuke Oba, Hiroki Furuta, Naoaki Okazaki
Discrete diffusion language models (DDLMs) generate text by iteratively denoising categorical token sequences, while recent drifting methods for continuous generators suggest that…
Diffusion-State Policy Optimization for Masked Diffusion Language Models
Daisuke Oba, Hiroki Furuta, Naoaki Okazaki
Masked diffusion language models generate text through iterative masked-token filling, but terminal-only rewards on final completions provide coarse credit assignment for the inter…
Stopping Computation for Converged Tokens in Masked Diffusion-LM Decoding
Daisuke Oba, Danushka Bollegala, Masahiro Kaneko +1
Masked Diffusion Language Models generate sequences via iterative sampling that progressively unmasks tokens. However, they still recompute the attention and feed-forward blocks fo…
JUBAKU: An Adversarial Benchmark for Exposing Culturally Grounded Stereotypes in Japanese LLMs
Taihei Shiotani, Masahiro Kaneko, Ayana Niwa +4
Social biases reflected in language are inherently shaped by cultural norms, which vary significantly across regions and lead to diverse manifestations of stereotypes. Existing eva…