4 papers
ZAYA1-8B Technical Report
Robert Washbourne, Rishi Iyer, Tomas Figliolia +15
We present ZAYA1-8B, a reasoning-focused mixture-of-experts (MoE) model with 700M active and 8B total parameters, built on Zyphra's MoE++ architecture. ZAYA1-8B's core pretraining,…
Embodied Referring Expression Comprehension in Human-Robot Interaction
Md Mofijul Islam, Alexi Gladstone, Sujan Sarker +5
As robots enter human workspaces, there is a crucial need for them to comprehend embodied human instructions, enabling intuitive and fluent human-robot interaction (HRI). However,…
Energy-Based Transformers are Scalable Learners and Thinkers
Alexi Gladstone, Ganesh Nanduru, Md Mofijul Islam +7
Inference-time computation techniques, analogous to human System 2 Thinking, have recently become popular for improving model performances. However, most existing approaches suffer…
Cognitively Inspired Energy-Based World Models
Alexi Gladstone, Ganesh Nanduru, Md Mofijul Islam +3
One of the predominant methods for training world models is autoregressive prediction in the output space of the next element of a sequence. In Natural Language Processing (NLP), t…