23 citations · 41 across the 11 of their papers we have counts for
11 papers
Principles and Practice of Deep Representation Learning: or a Mathematical Theory of Memory
Sam Buchanan, Druv Pai, Peng Wang +1
In the current era of deep learning and especially generative models, there is significant investment in training very large deep neural networks. Thus far, such models have been "…
On the Edge of Memorization in Diffusion Models
Sam Buchanan, Druv Pai, Yi Ma +1
When do diffusion models reproduce their training data, and when are they able to generate samples beyond it? A practically relevant theoretical understanding of this interplay bet…
Masked Completion via Structured Diffusion with White-Box Transformers
Druv Pai, Ziyang Wu, Sam Buchanan +2
Modern learning frameworks often train deep neural networks with massive amounts of unlabeled data to learn representations by solving simple pretext tasks, then use the representa…
White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
Yaodong Yu, Sam Buchanan, Druv Pai +7
In this paper, we contend that a natural objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a low-dimension…
Emergence of Segmentation with Minimalistic White-Box Transformers
Yaodong Yu, Tianzhe Chu, Shengbang Tong +4
Transformer-like models for vision tasks have recently proven effective for a wide range of downstream applications such as segmentation and detection. Previous works have shown th…
Canonical Factors for Hybrid Neural Fields
Brent Yi, Weijia Zeng, Sam Buchanan +1
Factored feature volumes offer a simple way to build more compact, efficient, and intepretable neural fields, but also introduce biases that are not necessarily beneficial for real…
White-Box Transformers via Sparse Rate Reduction
Yaodong Yu, Sam Buchanan, Druv Pai +5
In this paper, we contend that the objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a mixture of low-dime…
Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization
Sam Buchanan, Jingkai Yan, Ellie Haber +1
Achieving invariance to nuisance transformations is a fundamental challenge in the construction of robust and reliable vision systems. Existing approaches to invariance scale expon…
Deep Networks Provably Classify Data on Curves
Tingran Wang, Sam Buchanan, Dar Gilboa +1
Data with low-dimensional nonlinear structure are ubiquitous in engineering and scientific problems. We study a model problem with such structure -- a binary classification task th…
Deep Networks and the Multiple Manifold Problem
Sam Buchanan, Dar Gilboa, John Wright
We study the multiple manifold problem, a binary classification task modeled on applications in machine vision, in which a deep fully-connected neural network is trained to separat…
Efficient Dictionary Learning with Gradient Descent
Dar Gilboa, Sam Buchanan, John Wright
Randomly initialized first-order optimization algorithms are the method of choice for solving many high-dimensional nonconvex problems in machine learning, yet general theoretical…