7 papers
Prime Agent: A Self-Improving RLM Harness
Seth Karten, Alex L. Zhang, Kevin Thomas +8
Language models are sequential processors, but long-horizon agency requires external information and computation beyond model weights and active context. Prime Agent is an open-sou…
INTELLECT-3: Technical Report
Prime Intellect Team, Mika Senghaas, Fares Obeid +20
We present INTELLECT-3, a 106B-parameter Mixture-of-Experts model (12B active) trained with large-scale reinforcement learning on our end-to-end RL infrastructure stack. INTELLECT-…
Value Residual Learning
Zhanchao Zhou, Tianyi Wu, Zhiyun Jiang +2
While Transformer models have achieved remarkable success in various domains, the effectiveness of information propagation through deep networks remains a critical challenge. Stand…
INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning
Prime Intellect Team, Sami Jaghouar, Justus Mattern +11
We introduce INTELLECT-2, the first globally distributed reinforcement learning (RL) training run of a 32 billion parameter language model. Unlike traditional centralized training…
INTELLECT-1 Technical Report
Sami Jaghouar, Jack Min Ong, Manveer Basra +9
In this report, we introduce INTELLECT-1, the first 10 billion parameter language model collaboratively trained across the globe, demonstrating that large-scale model training is n…
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
Bo Peng, Daniel Goldstein, Quentin Anthony +27
We present Eagle (RWKV-5) and Finch (RWKV-6), sequence models improving upon the RWKV (RWKV-4) architecture. Our architectural design advancements include multi-headed matrix-value…