6 papers
LLM-as-an-Improver: Turning Verification into Better Candidates
Akiyoshi Tomihari, Yuma Ichikawa
Verifier-based selection improves LLM performance by generating multiple candidate solutions and using a verifier to select the most promising one. However, existing methods typica…
Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation
Akiyoshi Tomihari, Issei Sato
Recent analyses question whether reinforcement learning (RL) is responsible for strong reasoning in large language models (LLMs). At the same time, distillation and inference-time…
Learning Dynamics in RL Post-Training for Language Models
Akiyoshi Tomihari
Reinforcement learning (RL) post-training is a critical stage in modern language model development, playing a key role in improving alignment and reasoning ability. However, severa…
Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians
Akiyoshi Tomihari, Ryo Karakida
The theoretical understanding of self-attention (SA) has been steadily progressing. A prominent line of work studies a class of SA layers that admit an energy function decreased by…
Gradient Heterogeneity Complements Hessian Heterogeneity in Transformer Optimization
Akiyoshi Tomihari, Issei Sato
Transformers are difficult to optimize with stochastic gradient descent (SGD) and largely rely on adaptive optimizers such as Adam. Despite extensive efforts, the mechanisms behind…
Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective
Akiyoshi Tomihari, Issei Sato
The two-stage fine-tuning (FT) method, linear probing (LP) then fine-tuning (LP-FT), outperforms linear probing and FT alone. This holds true for both in-distribution (ID) and out-…