6 papers
How Does Chain-of-Thought Supervision Teach Models to Reason? A Theoretical Analysis
Kaito Takanami, Cengiz Pehlevan
Training with chain-of-thought (CoT) supervision is an effective approach for teaching models multi-step reasoning procedures. At inference time, however, models may need to contin…
Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data
Kaito Takanami, Takashi Takahashi, Yoshiyuki Kabashima
Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture this behavior, yet evaluating lear…
An Asymptotic Theory of Chain-of-Thought in In-Context Learning
Kaito Takanami, Cengiz Pehlevan
Chain-of-thought (CoT) reasoning has become a widely used mechanism for eliciting multi-step reasoning in large language models by generating intermediate reasoning steps at infere…
Learning Linear Regression with Low-Rank Tasks in-Context
Kaito Takanami, Takashi Takahashi, Yoshiyuki Kabashima
In-context learning (ICL) is a key building block of modern large language models, yet its theoretical mechanisms remain poorly understood. It is particularly mysterious how ICL op…
The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model
Kaito Takanami, Takashi Takahashi, Ayaka Sakata
Self-distillation (SD), a technique where a model improves itself using its own predictions, has attracted attention as a simple yet powerful approach in machine learning. Despite…
Detection of diffusion anisotropy from an individual short particle trajectory
Kaito Takanami, Daisuke Taniguchi, Sawako Enoki +3
In parallel with advances in microscale imaging techniques, the fields of biology and materials science have focused on precisely extracting particle properties based on their diff…