507 citations · 672 across the 27 of their papers we have counts for
7 papers · 1 filter
Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aakshita Chandiramani +544
We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemo…
Learning Generative Selection for Best-of-N
Shubham Toshniwal, Aleksander Ficek, Siddhartha Jain +5
Scaling test-time compute via parallel sampling can substantially improve LLM reasoning, but is often limited by Best-of-N selection quality. Generative selection methods, such as…
GenSelect: A Generative Approach to Best-of-N
Shubham Toshniwal, Ivan Sorokin, Aleksander Ficek +2
Generative reward models with parallel sampling have enabled effective test-time scaling for reasoning tasks. Current approaches employ pointwise scoring of individual solutions or…
NeMo-Inspector: A Visualization Tool for LLM Generation Analysis
Daria Gitman, Igor Gitman, Evelina Bakhturina
Adapting Large Language Models (LLMs) to novel tasks and enhancing their overall capabilities often requires large, high-quality training datasets. Synthetic data, generated at sca…
Understanding the Role of Momentum in Stochastic Gradient Methods
Igor Gitman, Hunter Lang, Pengchuan Zhang +1
The use of momentum in stochastic gradient methods has become a widespread practice in machine learning. Different variants of momentum, including heavy-ball momentum, Nesterov's a…
Novel Prediction Techniques Based on Clusterwise Linear Regression
Igor Gitman, Jieshi Chen, Eric Lei +1
In this paper we explore different regression models based on Clusterwise Linear Regression (CLR). CLR aims to find the partition of the data into clusters, such that linear re…