1 citations · 1 across the 5 of their papers we have counts for
4 papers · 1 filter
Bayesian Wind Tunnels for Model Selection
Siddhartha R Dalal, Vishal Misra, Abhay Parekh
Prior work has shown that transformers can perform exact Bayesian filtering within a fixed hypothesis class. Can they also perform Bayesian model selection -- identifying the corre…
Geometric Scaling of Bayesian Inference in LLMs
Naman Agarwal, Siddhartha R. Dalal, Vishal Misra
Recent work has shown that small transformers trained in controlled "wind-tunnel'' settings can implement exact Bayesian inference, and that their training dynamics produce a geome…
The Bayesian Geometry of Transformer Attention
Naman Agarwal, Siddhartha R. Dalal, Vishal Misra
Transformers often appear to perform Bayesian reasoning in context, but verifying this rigorously has been impossible: natural data lack analytic posteriors, and large models confl…
Beyond the Black Box: A Statistical Model for LLM Reasoning and Inference
Siddhartha Dalal, Vishal Misra
This paper introduces a novel Bayesian learning model to explain the behavior of Large Language Models (LLMs), focusing on their core optimization metric of next token prediction.…