296 citations · 604 across the 20 of their papers we have counts for
22 papers
Training Transformers as a Universal Computer
Ruize Xu, Chenxiao Yang, Yanhong Li +1
We demonstrate that a small transformer can learn to execute programs in MicroPy, a simplified yet computationally universal programming language. Given procedure definitions toget…
On the Mathematics of Diffusion Models
David McAllester
This paper gives direct derivations of the differential equations and likelihood formulas of diffusion models assuming only knowledge of Gaussian distributions. A VAE analysis deri…
Dependent Type Theory as Related to the Bourbaki Notions of Structure and Isomorphism
David McAllester
This paper develops a version of dependent type theory in which isomorphism is handled through a direct generalization of the 1939 definitions of Bourbaki. More specifically we gen…
Information-Theoretic Segmentation by Inpainting Error Maximization
Pedro Savarese, Sunnie S. Y. Kim, Michael Maire +2
We study image segmentation from an information-theoretic perspective, proposing a novel adversarial method that performs unsupervised segmentation by partitioning images into maxi…
On-The-Fly Information Retrieval Augmentation for Language Models
Hai Wang, David McAllester
Here we experiment with the use of information retrieval as an augmentation for pre-trained language models. The text corpus used in information retrieval can be viewed as form of…
MathZero, The Classification Problem, and Set-Theoretic Type Theory
David McAllester
AlphaZero learns to play go, chess and shogi at a superhuman level through self play given only the rules of the game. This raises the question of whether a similar thing could be…