223 citations · 321 across the 16 of their papers we have counts for
16 papers
NVIDIA Nemotron 3: Efficient and Open Intelligence
NVIDIA, :, Aaron Blakeman +356
We introduce the Nemotron 3 family of models - Nano, Super, and Ultra. These models deliver strong agentic, reasoning, and conversational capabilities. The Nemotron 3 family uses a…
Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aaron Blakeman +311
We present Nemotron 3 Nano 30B-A3B, a Mixture-of-Experts hybrid Mamba-Transformer language model. Nemotron 3 Nano was pretrained on 25 trillion text tokens, including more than 3 t…
Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
Yonggan Fu, Xin Dong, Shizhe Diao +12
Efficient deployment of small language models (SLMs) is essential for numerous real-world applications with stringent latency constraints. While previous work on SLM design has pri…
Pruning by Active Attention Manipulation
Zahra Babaiee, Lucas Liebenwein, Ramin Hasani +2
Filter pruning of a CNN is typically achieved by applying discrete masks on the CNN's filter weights or activation maps, post-training. Here, we present a new filter-importance-sco…
End-to-End Sensitivity-Based Filter Pruning
Zahra Babaiee, Lucas Liebenwein, Ramin Hasani +2
In this paper, we present a novel sensitivity-based filter pruning algorithm (SbF-Pruner) to learn the importance scores of filters of each layer end-to-end. Our method learns the…
Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition
Lucas Liebenwein, Alaa Maalouf, Oren Gal +2
We present a novel global compression framework for deep neural networks that automatically analyzes each layer to identify the optimal per-layer compression ratio, while simultane…
Closed-form Continuous-time Neural Models
Ramin Hasani, Mathias Lechner, Alexander Amini +5
Continuous-time neural processes are performant sequential decision-makers that are built by differential equations (DE). However, their expressive power when they are deployed on…
Sparse Flows: Pruning Continuous-depth Models
Lucas Liebenwein, Ramin Hasani, Alexander Amini +1
Continuous deep learning architectures enable learning of flexible probabilistic models for predictive modeling as neural ordinary differential equations (ODEs), and for generative…
Low-Regret Active learning
Cenk Baykal, Lucas Liebenwein, Dan Feldman +1
We develop an online learning algorithm for identifying unlabeled data points that are most informative for training (i.e., active learning). By formulating the active learning pro…
Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy
Lucas Liebenwein, Cenk Baykal, Brandon Carter +2
Neural network pruning is a popular technique used to reduce the inference costs of modern, potentially overparameterized, networks. Starting from a pre-trained network, the proces…
Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space
Wilko Schwarting, Tim Seyde, Igor Gilitschenski +4
Learning competitive behaviors in multi-agent settings such as racing requires long-term reasoning about potential adversarial interactions. This paper presents Deep Latent Competi…
Machine Learning-based Estimation of Forest Carbon Stocks to increase Transparency of Forest Preservation Efforts
Björn Lütjens, Lucas Liebenwein, Katharina Kramer
An increasing amount of companies and cities plan to become CO2-neutral, which requires them to invest in renewable energies and carbon emission offsetting solutions. One of the ch…
Provable Filter Pruning for Efficient Neural Networks
Lucas Liebenwein, Cenk Baykal, Harry Lang +2
We present a provable, sampling-based approach for generating compact Convolutional Neural Networks (CNNs) by identifying and removing redundant filters from an over-parameterized…
SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks
Cenk Baykal, Lucas Liebenwein, Igor Gilitschenski +2
We introduce a pruning algorithm that provably sparsifies the parameters of a trained model in a way that approximately preserves the model's predictive accuracy. Our algorithm use…
Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds
Cenk Baykal, Lucas Liebenwein, Igor Gilitschenski +2
We present an efficient coresets-based neural network compression algorithm that sparsifies the parameters of a trained fully-connected neural network in a manner that provably app…
Training Support Vector Machines using Coresets
Cenk Baykal, Lucas Liebenwein, Wilko Schwarting
We present a novel coreset construction algorithm for solving classification tasks using Support Vector Machines (SVMs) in a computationally efficient manner. A coreset is a weight…