2 citations · 3 across the 5 of their papers we have counts for
3 papers · 1 filter
Native Multi-Dimensional Subquadratic Operators via Input Dependent Long Convolutions
David R. Wessels, Farhad Ramezanghorbani, Alireza Moradzadeh +9
Subquadratic alternatives to attention require compromises when applied to multi-dimensional data: standard convolutions lack global receptive fields and input dependency, while re…
PairSAE: Mechanistic Interpretability from Pair Representations in Protein Co-Folding
Giosue Migliorini, Aristofanis Rontogiannis, Grigori Guitchounts +3
Foundation models for structural biology have achieved remarkable performance in predicting biomolecular structure and show promise for the design of proteins and small molecules.…
Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aaron Blakeman +572
We introduce Nemotron 3 Ultra, a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model. We pre-trained Nemotron 3 Ultra on 20 t…