1 citations · 1 across the 3 of their papers we have counts for
3 papers
cs.CL2026
Suppressing Final Layer Hidden State Jumps in Transformer Pretraining
Keigo Shibata, Kazuki Yano, Ryosuke Takahashi +3
This paper discusses the internal behavior of Transformer language models. Many recent pre-trained models have been reported to exhibit only slight changes in the angular distance…
cs.CL2025
An Open and Reproducible Deep Research Agent for Long-Form Question Answering
Ikuya Yamada, Wataru Ikeda, Ko Yoshida +5
We present an open deep research system for long-form question answering, selected as a winning system in the text-to-text track of the MMU-RAG competition at NeurIPS 2025. The sys…
cs.CL2025★ 1 cited
Layerwise Importance Analysis of Feed-Forward Networks in Transformer-based Language Models
Wataru Ikeda, Kazuki Yano, Ryosuke Takahashi +3
This study investigates the layerwise importance of feed-forward networks (FFNs) in Transformer-based language models during pretraining. We introduce an experimental approach that…