8 papers
Fix Initial Programs and Iteratively Refine Repair Instructions Toward Non-Elimination Multi-Turn Program Correction
Yuto Tanaka, Issei Sato
Recent work on large language models (LLMs) has emphasized the importance of scaling inference compute. From this perspective, the state-of-the-art method Scattered Forest Search (…
Max-pooling Network Revisited: Analyzing the Role of Semantic Probability in Multiple Instance Learning for Hallucination Detection
Shota Fujikawa, Issei Sato
Hallucination detection has become increasingly important for improving the reliability of large language models (LLMs). Recently, hybrid approaches such as HaMI, which combine sem…
A Formal Comparison Between Chain of Thought and Latent Thought
Kevin Xu, Issei Sato
Chain of thought (CoT) elicits reasoning in large language models by explicitly generating intermediate tokens. In contrast, latent thought reasoning operates directly in the conti…
Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation
Akiyoshi Tomihari, Issei Sato
Recent analyses question whether reinforcement learning (RL) is responsible for strong reasoning in large language models (LLMs). At the same time, distillation and inference-time…
Gradient Heterogeneity Complements Hessian Heterogeneity in Transformer Optimization
Akiyoshi Tomihari, Issei Sato
Transformers are difficult to optimize with stochastic gradient descent (SGD) and largely rely on adaptive optimizers such as Adam. Despite extensive efforts, the mechanisms behind…
To CoT or To Loop? A Formal Comparison Between Chain-of-Thought and Looped Transformers
Kevin Xu, Issei Sato
Chain-of-Thought (CoT) and Looped Transformers have been shown to empirically improve performance on reasoning tasks and to theoretically enhance expressivity by recursively increa…