5 papers · 1 filter
A Theory of Generalization in Deep Learning
Elon Litman, Gabe Guo
We present a non-asymptotic theory of generalization in deep learning where the empirical neural tangent kernel partitions the output space. In directions corresponding to signal,…
ABC: Any-Subset Autoregression via Non-Markovian Diffusion Bridges in Continuous Time and Space
Gabe Guo, Thanawat Sornwanee, Lutong Hao +3
Generating continuous-time, continuous-space stochastic processes (e.g., videos, weather forecasts) conditioned on partial observations (e.g., first and last frames) is a fundament…
You Need Better Attention Priors
Elon Litman, Gabe Guo
We generalize the attention mechanism by viewing it through the lens of Entropic Optimal Transport, revealing that standard attention corresponds to a transport problem regularized…
Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding
Gabe Guo, Stefano Ermon
In arbitrary-order language models, it is an open question how to sample tokens in parallel from the correct joint distribution. With discrete diffusion models, the more tokens the…
Sequencing the Neurome: Towards Scalable Exact Parameter Reconstruction of Black-Box Neural Networks
Judah Goldfeder, Quinten Roets, Gabe Guo +2
Inferring the exact parameters of a neural network with only query access is an NP-Hard problem, with few practical existing algorithms. Solutions would have major implications for…