103 citations · 187 across the 25 of their papers we have counts for
6 papers · 1 filter
Drifting Objectives for Refining Discrete Diffusion Language Models
Daisuke Oba, Hiroki Furuta, Naoaki Okazaki
Discrete diffusion language models (DDLMs) generate text by iteratively denoising categorical token sequences, while recent drifting methods for continuous generators suggest that…
Diffusion-State Policy Optimization for Masked Diffusion Language Models
Daisuke Oba, Hiroki Furuta, Naoaki Okazaki
Masked diffusion language models generate text through iterative masked-token filling, but terminal-only rewards on final completions provide coarse credit assignment for the inter…
Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
Gouki Minegishi, Hiroki Furuta, Shohei Taniguchi +2
Transformer-based language models exhibit In-Context Learning (ICL), where predictions are made adaptively based on context. While prior work links induction heads to ICL through a…
Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks
Lutfi Eren Erdogan, Nicholas Lee, Sehoon Kim +5
Large language models (LLMs) have shown remarkable advancements in enabling language agents to tackle simple tasks. However, applying them for complex, multi-step, long-horizon tas…
Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words
Gouki Minegishi, Hiroki Furuta, Yusuke Iwasawa +1
Sparse autoencoders (SAEs) have gained a lot of attention as a promising tool to improve the interpretability of large language models (LLMs) by mapping the complex superposition o…
A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta +2
Current Large Language Models (LLMs) are not only limited to some maximum context length, but also are not able to robustly consume long inputs. To address these limitations, we pr…