119 citations · 515 across the 60 of their papers we have counts for
13 papers · 1 filter
CRT-HMAR: Causal Requirement Tracing-Guided Hierarchical Multi-Agent Regulation for Open-Task-Aware Infrared-Visible Image Fusion
Zengyi Yang, Shuai Yuan, Zhong-Cheng Wu +3
Infrared and visible (IR-VIS) image fusion integrates complementary multimodal information into a single fused image to support downstream vision tasks. However, existing methods a…
How Local Mixing Encodes Relative Position in Global NoPE Attention
Cutter Dawes, Nick Alonso, Tom Figliolia +1
The attention operation is naively position invariant. However, positional information is fundamental to natural language, and therefore a variety of explicit position encodings ha…
PUFFER: Incremental Fuzzy Deduplication for Continuously Evolving Corpora
Xiao Yang, Erik Edward Aldape, Beren Millidge
Large language model training corpora grow through successive, often redundant releases, so each release must be deduplicated against both itself and the accumulated history. At tr…
ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution
Christopher Warner, Jonas Mago, JR Huml +1
We introduce ZUNA1.1, a 380M-parameter diffusion autoencoder for flexible EEG signal reconstruction. ZUNA1.1 is capable of reconstructing variable length sequences of up to 30s, wi…
ZONOS2 Technical Report
Gabriel Clark, Sofian Mejjoute, Mohamed Osman +2
We present ZONOS2 8B, our latest TTS model, which achieves state-of-the-art naturalness, prosody, and voice cloning fidelity. We improve upon Zonos-v0.1 across scale, data, and tra…
Can Scale Save Us From Plasticity Loss in Large Language Models?
J. Fernando Hernandez-Garcia, Tomás Figliolia, Beren Millidge
The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural…
Scaling Adaptive Depth with Norm-Agnostic Residual Networks
Tomás Figliolia, Beren Millidge
Residual architectures are ubiquitous in deep learning, but they suffer from a subtle structural limitation: the norm of the residual stream can grow rapidly with depth. As a resul…
Zamba2-VL Technical Report
Hassan Shapourian, Kasra Hejazi, Olabode M. Sule +1
We present Zamba2-VL, a suite of vision-language models built on Zamba2, a hybrid language-model architecture combining Mamba2 state-space layers with a small number of shared tran…
ZAYA1-VL-8B Technical Report
Hassan Shapourian, Kasra Hejazi, Olabode M. Sule +1
We present ZAYA1-VL-8B, a compact mixture-of-experts vision-language model built upon our in-house language model, ZAYA1-8B. Despite its compact size, ZAYA1-VL achieves performance…
ZAYA1-8B Technical Report
Robert Washbourne, Rishi Iyer, Tomas Figliolia +15
We present ZAYA1-8B, a reasoning-focused mixture-of-experts (MoE) model with 700M active and 8B total parameters, built on Zyphra's MoE++ architecture. ZAYA1-8B's core pretraining,…
Hybrid Associative Memories
Leon Lufkin, Tomás Figliolia, Beren Millidge +1
Recurrent neural networks (RNNs) and self-attention are both widely used sequence-mixing layers that maintain an internal memory. However, this memory is constructed using two orth…
ZUNA: Flexible EEG Superresolution with Position-Aware Diffusion Autoencoders
Christopher Warner, Jonas Mago, JR Huml +2
We present \texttt{ZUNA}, a 380M-parameter masked diffusion autoencoder trained to perform masked channel infilling and superresolution for arbitrary electrode numbers and position…
Online Vector Quantized Attention
Nick Alonso, Tomas Figliolia, Beren Millidge
Standard sequence mixing layers used in language models struggle to balance efficiency and performance. Self-attention performs well on long context tasks but has expensive quadrat…