23 citations · 48 across the 16 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Nested Sparsity for Scalable Pretraining and Continual Learning
Tianzhe Chu, Druv Pai, Yi Ma
Mixture-of-experts (MoE) transformers, the standard backbone for modern large-scale deep learning, employ conditional computation: only a sparse subset of model weights is used in…
cs.LG2026
Principles and Practice of Deep Representation Learning: or a Mathematical Theory of Memory
Sam Buchanan, Druv Pai, Peng Wang +1
In the current era of deep learning and especially generative models, there is significant investment in training very large deep neural networks. Thus far, such models have been "…