Publications (7)
On Holder-Brascamp-Lieb inequalities for torsion-free discrete Abelian groups
Michael Christ, James Demmel, Nicholas Knight +2
Hölder-Brascamp-Lieb inequalities provide upper bounds for a class of multilinear expressions, in terms of norms of the functions involved. They have been extensively studie…
Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aaron Blakeman +571
We introduce Nemotron 3 Ultra, a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model. We pre-trained Nemotron 3 Ultra on 20 t…
Riemannian Gradient Descent for Low-Rank Architectures
Nicholas Knight
We explore Riemannian optimization techniques for rank-factored matrix parameters, targeting contemporary deep learning applications. We examine ten points in the algorithm design…
A 3D Parallel Algorithm for QR Decomposition
Grey Ballard, James Demmel, Laura Grigori +2
Interprocessor communication often dominates the runtime of large matrix computations. We present a parallel algorithm for computing QR decompositions whose bandwidth cost (communi…
Hypergraph Partitioning for Sparse Matrix-Matrix Multiplication
Grey Ballard, Alex Druinsky, Nicholas Knight +1
We propose a fine-grained hypergraph model for sparse matrix-matrix multiplication (SpGEMM), a key computational kernel in scientific computing and data analysis whose performance…
Communication lower bounds and optimal algorithms for programs that reference arrays -- Part 1
Michael Christ, James Demmel, Nicholas Knight +2
The movement of data (communication) between levels of a memory hierarchy, or between parallel processors on a network, can greatly dominate the cost of computation, so algorithms…