4 citations · 16 across the 9 of their papers we have counts for
9 papers
Tensor Decomposition Meets RKHS: Efficient Algorithms for Smooth and Misaligned Data
Brett W. Larsen, Tamara G. Kolda, Anru R. Zhang +1
The canonical polyadic (CP) tensor decomposition decomposes a multidimensional data array into a sum of outer products of finite-dimensional vectors. Instead, we can replace some o…
Does your data spark joy? Performance gains from domain upsampling at the end of training
Cody Blakeney, Mansheej Paul, Brett W. Larsen +2
Pretraining datasets for large language models (LLMs) have grown to trillions of tokens composed of large amounts of CommonCrawl (CC) web scrape along with smaller, domain-specific…
Duality of Bures and Shape Distances with Implications for Comparing Neural Representations
Sarah E. Harvey, Brett W. Larsen, Alex H. Williams
A multitude of (dis)similarity measures between neural network representations have been proposed, resulting in a fragmented research landscape. Most of these measures fall into on…
Estimating Shape Distances on Neural Representations with Limited Samples
Dean A. Pospisil, Brett W. Larsen, Sarah E. Harvey +1
Measuring geometric similarity between high-dimensional network representations is a topic of longstanding interest to neuroscience and deep learning. Although many methods have be…
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?
Mansheej Paul, Feng Chen, Brett W. Larsen +3
Modern deep learning involves training costly, highly overparameterized networks, thus motivating the search for sparser networks that can still be trained to the same accuracy as…
Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks
Mansheej Paul, Brett W. Larsen, Surya Ganguli +2
A striking observation about iterative magnitude pruning (IMP; Frankle et al. 2020) is that $\unicode{x2014}$ after just a few hundred steps of dense training $\unicode{x2014}$ the…
Revisiting Approximate Leverage Score Sketching for Matrix Least Squares
Brett W. Larsen, Tamara G. Kolda
We revisit the problem of sketching using approximate leverage scores for matrix least squares problems of the form where the design matrix $A \in \mathbb{R}^{N…
How many degrees of freedom do we need to train deep networks: a loss landscape perspective
Brett W. Larsen, Stanislav Fort, Nic Becker +1
A variety of recent works, spanning pruning, lottery tickets, and training within random subspaces, have shown that deep neural networks can be trained using far fewer degrees of f…
Avoiding Spurious Local Minima in Deep Quadratic Networks
Abbas Kazemipour, Brett W. Larsen, Shaul Druckmann
Despite their practical success, a theoretical understanding of the loss landscape of neural networks has proven challenging due to the high-dimensional, non-convex, and highly non…