44 citations · 92 across the 14 of their papers we have counts for
14 papers
Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini
Madhuri Shanbhogue, Zhe Li, Shanfeng Zhang +86
We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified representation space. We leverage t…
MedASR: An Open-Source Model for High-Accuracy Medical Dictation
Ke Wu, Ehsan Variani, Tom Bagby +2
We present MedASR, an open-source 105M-parameter model engineered for high-accuracy medical dictation. Prioritizing a "small, fast, and accurate" design, MedASR addresses 3 core pi…
Benchmarking LLMs on the Massive Sound Embedding Benchmark (MSEB)
Cyril Allauzen, Tom Bagby, Georg Heigold +2
The Massive Sound Embedding Benchmark (MSEB) has emerged as a standard for evaluating the functional breadth of audio models. While initial baselines focused on specialized encoder…
Massive Sound Embedding Benchmark (MSEB)
Georg Heigold, Ehsan Variani, Tom Bagby +4
Audio is a critical component of multimodal perception, and any truly intelligent system must demonstrate a wide range of auditory capabilities. These capabilities include transcri…
SequenceLayers: Sequence Processing and Streaming Neural Networks Made Easy
RJ Skerry-Ryan, Julian Salazar, Soroosh Mariooryad +8
We introduce a neural network layer API and library for sequence modeling, designed for easy creation of sequence models that can be executed both layer-by-layer (e.g., teacher-for…
LAST: Scalable Lattice-Based Speech Modelling in JAX
Ke Wu, Ehsan Variani, Tom Bagby +1
We introduce LAST, a LAttice-based Speech Transducer library in JAX. With an emphasis on flexibility, ease-of-use, and scalability, LAST implements differentiable weighted finite s…
Learning the joint distribution of two sequences using little or no paired data
Soroosh Mariooryad, Matt Shannon, Siyuan Ma +5
We present a noisy channel generative model of two sequences, for example text and speech, which enables uncovering the association between the two modalities when limited paired d…
Speaker Generation
Daisy Stanton, Matt Shannon, Soroosh Mariooryad +4
This work explores the task of synthesizing speech in nonexistent human-sounding voices. We call this task "speaker generation", and present TacoSpawn, a system that performs compe…
Non-saturating GAN training as divergence minimization
Matt Shannon, Ben Poole, Soroosh Mariooryad +5
Non-saturating generative adversarial network (GAN) training is widely used and has continued to obtain groundbreaking results. However so far this approach has lacked strong theor…
Semi-Supervised Generative Modeling for Controllable Speech Synthesis
Raza Habib, Soroosh Mariooryad, Matt Shannon +5
We present a novel generative model that combines state-of-the-art neural text-to-speech (TTS) with semi-supervised probabilistic latent variable models. By providing partial super…
Location-Relative Attention Mechanisms For Robust Long-Form Speech Synthesis
Eric Battenberg, RJ Skerry-Ryan, Soroosh Mariooryad +4
Despite the ability to produce human-level speech for in-domain text, attention-based end-to-end text-to-speech (TTS) systems suffer from text alignment failures that increase in f…
Complex Evolution Recurrent Neural Networks (ceRNNs)
Izhak Shafran, Tom Bagby, R. J. Skerry-Ryan
Unitary Evolution Recurrent Neural Networks (uRNNs) have three attractive properties: (a) the unitary property, (b) the complex-valued nature, and (c) their efficient linear operat…
Effective Use of Variational Embedding Capacity in Expressive End-to-End Speech Synthesis
Eric Battenberg, Soroosh Mariooryad, Daisy Stanton +4
Recent work has explored sequence-to-sequence latent variable models for expressive speech synthesis (supporting control and transfer of prosody and style), but has not presented a…
Streaming End-to-end Speech Recognition For Mobile Devices
Yanzhang He, Tara N. Sainath, Rohit Prabhavalkar +17
End-to-end (E2E) models, which directly predict output character sequences given input speech, are good candidates for on-device speech recognition. E2E models, however, present nu…