60 citations · 87 across the 19 of their papers we have counts for
21 papers
Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse
Taekyung Heo, Rasoul Shafipour, Ritchie Zhao +6
Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver…
Directed Graph Topology Inference via Graph Filter Identification
Rasoul Shafipour, Andrei Buciulea, Santiago Segarra +2
We address the problem of inferring a directed network from nodal measurements generated by linear diffusion dynamics on the sought graph. Observations are modeled as the outputs o…
Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aaron Blakeman +572
We introduce Nemotron 3 Ultra, a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model. We pre-trained Nemotron 3 Ultra on 20 t…
Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
Hayate Iso, Tiyasa Mitra, Sudipta Mondal +15
RL post-training of frontier language models is increasingly bottlenecked by autoregressive rollout generation, making rollout acceleration a central systems challenge. Many existi…
Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aakshita Chandiramani +544
We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemo…
LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts
Venmugil Elango, Nidhi Bhatia, Roger Waleffe +13
Mixture of Experts (MoEs) have become a central component of many state-of-the-art open-source and proprietary large language models. Despite their widespread adoption, it remains…
NVIDIA Nemotron 3: Efficient and Open Intelligence
NVIDIA, :, Aaron Blakeman +356
We introduce the Nemotron 3 family of models - Nano, Super, and Ultra. These models deliver strong agentic, reasoning, and conversational capabilities. The Nemotron 3 family uses a…
Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aaron Blakeman +311
We present Nemotron 3 Nano 30B-A3B, a Mixture-of-Experts hybrid Mamba-Transformer language model. Nemotron 3 Nano was pretrained on 25 trillion text tokens, including more than 3 t…
SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
Rasoul Shafipour, David Harrison, Maxwell Horton +6
Large Language Models (LLMs) have transformed natural language processing, but face significant challenges in widespread deployment due to their high runtime cost. In this paper, w…
Microscaling Data Formats for Deep Learning
Bita Darvish Rouhani, Ritchie Zhao, Ankit More +30
Narrow bit-width data formats are key to reducing the computational and storage costs of modern deep learning applications. This paper evaluates Microscaling (MX) data formats that…
With Shared Microexponents, A Little Shifting Goes a Long Way
Bita Rouhani, Ritchie Zhao, Venmugil Elango +19
This paper introduces Block Data Representations (BDR), a framework for exploring and evaluating a wide spectrum of narrow-precision formats for deep learning. It enables compariso…
Non-Local Feature Aggregation on Graphs via Latent Fixed Data Structures
Mostafa Rahmani, Rasoul Shafipour, Ping Li
In contrast to image/text data whose order can be used to perform non-local feature aggregation in a straightforward way using the pooling layers, graphs lack the tensor representa…
Online Topology Inference from Streaming Stationary Graph Signals with Partial Connectivity Information
Rasoul Shafipour, Gonzalo Mateos
We develop online graph learning algorithms from streaming network data. Our goal is to track the (possibly) time-varying network topology, and effect memory and computational savi…
A Novel Scheme for Support Identification and Iterative Sampling of Bandlimited Graph Signals
Abolfazl Hashemi, Rasoul Shafipour, Haris Vikalo +1
We study the problem of sampling and reconstruction of bandlimited graph signals where the objective is to select a node subset of prescribed cardinality that ensures interpolation…
Towards Accelerated Greedy Sampling and Reconstruction of Bandlimited Graph Signals
Abolfazl Hashemi, Rasoul Shafipour, Haris Vikalo +1
We study the problem of sampling and reconstructing spectrally sparse graph signals where the objective is to select a subset of nodes of prespecified cardinality that ensures inte…
A Directed Graph Fourier Transform with Spread Frequency Components
Rasoul Shafipour, Ali Khodabakhsh, Gonzalo Mateos +1
We study the problem of constructing a graph Fourier transform (GFT) for directed graphs (digraphs), which decomposes graph signals into different modes of variation with respect t…
Blind Identification of Invertible Graph Filters with Multiple Sparse Inputs
Chang Ye, Rasoul Shafipour, Gonzalo Mateos
This paper deals with problem of blind identification of a graph filter and its sparse input signal, thus broadening the scope of classical blind deconvolution of temporal and spat…
Identifying the Topology of Undirected Networks from Diffused Non-stationary Graph Signals
Rasoul Shafipour, Santiago Segarra, Antonio G. Marques +1
We address the problem of inferring an undirected graph from nodal observations, which are modeled as non-stationary graph signals generated by local diffusion dynamics that depend…
Sampling and Reconstruction of Graph Signals via Weak Submodularity and Semidefinite Relaxation
Abolfazl Hashemi, Rasoul Shafipour, Haris Vikalo +1
We study the problem of sampling a bandlimited graph signal in the presence of noise, where the objective is to select a node subset of prescribed cardinality that minimizes the si…
Buildup of Speaking Skills in an Online Learning Community: A Network-Analytic Exploration
Rasoul Shafipour, Raiyan Abdul Baten, Md Kamrul Hasan +3
In this study, we explore peer-interaction effects in online networks on speaking skill development. In particular, we present an evidence for gradual buildup of skills in a small-…
A Digraph Fourier Transform With Spread Frequency Components
Rasoul Shafipour, Ali Khodabakhsh, Gonzalo Mateos +1
We study the problem of constructing a graph Fourier transform (GFT) for directed graphs (digraphs), which decomposes graph signals into different modes of variation with respect t…