4 citations · 6 across the 15 of their papers we have counts for
7 papers · 1 filter
Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aakshita Chandiramani +544
We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemo…
Learning Generative Selection for Best-of-N
Shubham Toshniwal, Aleksander Ficek, Siddhartha Jain +5
Scaling test-time compute via parallel sampling can substantially improve LLM reasoning, but is often limited by Best-of-N selection quality. Generative selection methods, such as…
Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradients
Christos Thrampoulidis, Sadegh Mahdavi, Wenlong Deng
This note reconciles two seemingly distinct approaches to policy gradient optimization for the Pass@K objective in reinforcement learning with verifiable rewards: (1) direct REINFO…
From Graph Diffusion to Graph Classification
Jia Jun Cheng Xian, Sadegh Mahdavi, Renjie Liao +1
Generative models such as diffusion models have achieved remarkable success in state-of-the-art image and text tasks. Recently, score-based diffusion models have extended their suc…
Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Models
Sadegh Mahdavi, Raquel Aoki, Keyi Tang +1
Large Language Models (LLMs) have shown remarkable performance in various natural language tasks, but they often struggle with planning problems that require structured reasoning.…
Memorization Capacity of Multi-Head Attention in Transformers
Sadegh Mahdavi, Renjie Liao, Christos Thrampoulidis
Transformers have become the go-to architecture for language and vision tasks, yet their theoretical properties, especially memorization capacity, remain elusive. This paper invest…
Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks
Sadegh Mahdavi, Kevin Swersky, Thomas Kipf +3
In this paper, we study the OOD generalization of neural algorithmic reasoning tasks, where the goal is to learn an algorithm (e.g., sorting, breadth-first search, and depth-first…