Publications (37)
Changing Base Without Losing Pace: A GPU-Efficient Alternative to MatMul in DNNs
Nir Ailon, Akhiad Bercovich, Yahel Uffenheimer +1
Modern AI relies on huge matrix multiplications (MatMuls), whose computation poses a scalability problem for inference and training. We propose an alternative, GPU native bilinear…
NVIDIA Nemotron 3: Efficient and Open Intelligence
NVIDIA, :, Aaron Blakeman +356
We introduce the Nemotron 3 family of models - Nano, Super, and Ultra. These models deliver strong agentic, reasoning, and conversational capabilities. The Nemotron 3 family uses a…
Sparse Linear Networks with a Fixed Butterfly Structure: Theory and Practice
Nir Ailon, Omer Leibovich, Vineet Nair
A butterfly network consists of logarithmically many layers, each with a linear number of non-zero weights (pre-specified). The fast Johnson-Lindenstrauss transform (FJLT) can be r…
Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aaron Blakeman +571
We introduce Nemotron 3 Ultra, a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model. We pre-trained Nemotron 3 Ultra on 20 t…
An Improved Algorithm for Bipartite Correlation Clustering
Nir Ailon, Noa Avigdor-Elgrabli, Edo Liberty
Bipartite Correlation clustering is the problem of generating a set of disjoint bi-cliques on a set of nodes while minimizing the symmetric difference to a bipartite input graph. T…
Approximate Correlation Clustering Using Same-Cluster Queries
Nir Ailon, Anup Bhattacharya, Ragesh Jaiswal
Ashtiani et al. (NIPS 2016) introduced a semi-supervised framework for clustering (SSAC) where a learner is allowed to make same-cluster queries. More specifically, in their model,…
An n\log n Lower Bound for Fourier Transform Computation in the Well Conditioned Model
Nir Ailon
Obtaining a non-trivial (super-linear) lower bound for computation of the Fourier transform in the linear circuit model has been a long standing open problem for over 40 years. An…
A Lower Bound for Fourier Transform Computation in a Linear Model Over 2x2 Unitary Gates Using Matrix Entropy
Nir Ailon
Obtaining a non-trivial (super-linear) lower bound for computation of the Fourier transform in the linear circuit model has been a long standing open problem. All lower bounds so f…
Approximate Clustering with Same-Cluster Queries
Nir Ailon, Anup Bhattacharya, Ragesh Jaiswal +1
Ashtiani et al. proposed a Semi-Supervised Active Clustering framework (SSAC), where the learner is allowed to make adaptive queries to a domain expert. The queries are of the kind…
An Active Learning Algorithm for Ranking from Pairwise Preferences with an Almost Optimal Query Complexity
Nir Ailon
We study the problem of learning to rank from pairwise preferences, and solve a long-standing open problem that has led to development of many heuristics but no provable results fo…
Fast and RIP-optimal transforms
Nir Ailon, Holger Rauhut
We study constructions of matrices that both (1) satisfy the restricted isometry property (RIP) at sparsity with optimal parameters, and (2) are efficient in t…
Paraunitary Matrices, Entropy, Algebraic Condition Number and Fourier Computation
Nir Ailon
The Fourier Transform is one of the most important linear transformations used in science and engineering. Cooley and Tukey's Fast Fourier Transform (FFT) from 1964 is a method for…
A note on: No need to choose: How to get both a PTAS and Sublinear Query Complexity
Nir Ailon, Zohar Karnin
We revisit various PTAS's (Polynomial Time Approximation Schemes) for minimization versions of dense problems, and show that they can be performed with sublinear query complexity.…
A tight lower bound instance for k-means++ in constant dimension
Anup Bhattacharya, Ragesh Jaiswal, Nir Ailon
The k-means++ seeding algorithm is one of the most popular algorithms that is used for finding the initial centers when using the k-means heuristic. The algorithm is a simple s…
Online Ranking: Discrete Choice, Spearman Correlation and Other Feedback
Nir Ailon
Given a set of objects, an online ranking system outputs at each time step a full ranking of the set, observes a feedback of some form and suffers a loss. We study the sett…
Tighter Fourier Transform Complexity Tradeoffs
Nir Ailon
The Fourier Transform is one of the most important linear transformations used in science and engineering. Cooley and Tukey's Fast Fourier Transform (FFT) from 1964 is a method for…
Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aakshita Chandiramani +544
We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemo…
Almost Optimal Unrestricted Fast Johnson-Lindenstrauss Transform
Nir Ailon, Edo Liberty
The problems of random projections and sparse reconstruction have much in common and individually received much attention. Surprisingly, until now they progressed in parallel and r…
Active Learning Using Smooth Relative Regret Approximations with Applications
Nir Ailon, Ron Begleiter, Esther Ezra
The disagreement coefficient of Hanneke has become a central data independent invariant in proving active learning rates. It has been shown in various ways that a concept class wit…
Semi-supervised deep learning by metric embedding
Elad Hoffer, Nir Ailon
Deep networks are successfully used as classification models yielding state-of-the-art results when trained on a large number of labeled samples. These models, however, are usually…
A Simple Linear Ranking Algorithm Using Query Dependent Intercept Variables
Nir Ailon
The LETOR website contains three information retrieval datasets used as a benchmark for testing machine learning ideas for ranking. Algorithms participating in the challenge are re…
Interesting Open Problem Related to Complexity of Computing the Fourier Transform and Group Theory
Nir Ailon
The Fourier Transform is one of the most important linear transformations used in science and engineering. Cooley and Tukey's Fast Fourier Transform (FFT) from 1964 is a method for…
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs
Akhiad Bercovich, Tomer Ronen, Talor Abramovich +23
Large language models (LLMs) offer remarkable capabilities, yet their high inference costs restrict wider adoption. While increasing parameter counts improves accuracy, it also bro…
Self-Improving Algorithms
Nir Ailon, Bernard Chazelle, Kenneth L. Clarkson +3
We investigate ways in which an algorithm can improve its expected performance by fine-tuning itself automatically with respect to an unknown input distribution D. We assume here t…
Extending Puzzle for Mixture-of-Experts Reasoning Models with Application to GPT-OSS Acceleration
Akhiad Bercovich, Nir Ailon, Vladimir Anisimov +21
Reasoning-focused LLMs improve answer quality by generating longer reasoning traces, but the additional tokens dramatically increase serving cost, motivating inference optimization…
Deep unsupervised learning through spatial contrasting
Elad Hoffer, Itay Hubara, Nir Ailon
Convolutional networks have marked their place over the last few years as the best performing model for various visual tasks. They are, however, most suited for supervised learning…
Nemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMs
Akhiad Bercovich, Talor Abramovich, Daniel Afrimi +67
We present Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super optimized for interactive deployment. We designed the model to maximize server throughput under…
Efficient NTK using Dimensionality Reduction
Nir Ailon, Supratim Shit
Recently, neural tangent kernel (NTK) has been used to explain the dynamics of learning parameters of neural networks, at the large width limit. Quantitative analyses of NTK give r…
Spatial contrasting for deep unsupervised learning
Elad Hoffer, Itay Hubara, Nir Ailon
Convolutional networks have marked their place over the last few years as the best performing model for various visual tasks. They are, however, most suited for supervised learning…
The Complexity of Computing (Almost) Unitary Matrices With $\eps$-Copies of the Fourier Transform
Nir Ailon, Gal Yehuda
The complexity of computing the Fourier transform is a longstanding open problem. Very recently, Ailon (2013, 2014, 2015) showed in a collection of papers that, roughly speaking, a…
Bandit Online Optimization Over the Permutahedron
Nir Ailon, Kohei Hatano, Eiji Takimoto
The permutahedron is the convex polytope with vertex set consisting of the vectors for all permutations (bijections) over . We study a b…
Reducing Dueling Bandits to Cardinal Bandits
Nir Ailon, Thorsten Joachims, Zohar Karnin
We present algorithms for reducing the Dueling Bandits problem to the conventional (stochastic) Multi-Armed Bandits problem. The Dueling Bandits problem is an online model of learn…
Breaking the Small Cluster Barrier of Graph Clustering
Nir Ailon, Yudong Chen, Xu Huan
This paper investigates graph clustering in the planted cluster model in the presence of {\em small clusters}. Traditional results dictate that for an algorithm to provably correct…
An efficient reduction of ranking to classification
Nir Ailon, Mehryar Mohri
This paper describes an efficient reduction of the learning problem of ranking to binary classification. The reduction guarantees an average pairwise misranking regret of at most t…
Active Learning of Custering with Side Information Using $\eps$-Smooth Relative Regret Approximations
Nir Ailon, Ron Begleiter
Clustering is considered a non-supervised learning setting, in which the goal is to partition a collection of data points into disjoint clusters. Often a bound on the number of…
Torsion points on curves and common divisors of a^k-1 and b^k-1
Nir Ailon, Zeev Rudnick
We study the behavior of the greatest common divisor of a^k-1 and b^k-1, where a,b are fixed integers or polynomials, and k varies. In the integer case, we conjecture that when a a…
Deep metric learning using Triplet network
Elad Hoffer, Nir Ailon
Deep learning has proven itself as a successful set of models for learning useful semantic representations of data. These, however, are mostly implicitly learned as part of a class…