425 citations · 1.2k across the 64 of their papers we have counts for
19 papers · 1 filter
Formal Language Knowledge Corpus for Retrieval Augmented Generation
Majd Zayyad, Yossi Adi
The integration of retrieval-augmented techniques with LLMs has shown promise in improving performance across various domains. However, their utility in tasks requiring advanced re…
Enhancing TTS Stability in Hebrew using Discrete Semantic Units
Ella Zeldes, Or Tal, Yossi Adi
This study introduces a refined approach to Text-to-Speech (TTS) generation that significantly enhances sampling stability across languages, with a particular focus on Hebrew. By l…
DM-Codec: Distilling Multimodal Representations for Speech Tokenization
Md Mubtasim Ahasan, Md Fahim, Tasnim Mohiuddin +6
Recent advancements in speech-language models have yielded significant improvements in speech tokenization and synthesis. However, effectively mapping the complex, multidimensional…
LAST: Language Model Aware Speech Tokenization
Arnon Turetzky, Yossi Adi
Speech tokenization serves as the foundation of speech language model (LM), enabling them to perform various tasks such as spoken language modeling, text-to-speech, speech-to-text,…
Latent Watermarking of Audio Generative Models
Robin San Roman, Pierre Fernandez, Antoine Deleforge +2
The advancements in audio generative models have opened up new challenges in their responsible disclosure and the detection of their misuse. In response, we introduce a method to w…
Salmon: A Suite for Acoustic Language Model Evaluation
Gallil Maimon, Amit Roth, Yossi Adi
Speech language models have recently demonstrated great potential as universal speech processing systems. Such models have the ability to model the rich acoustic information existi…
Audio Enhancement from Multiple Crowdsourced Recordings: A Simple and Effective Baseline
Shiran Aziz, Yossi Adi, Shmuel Peleg
With the popularity of cellular phones, events are often recorded by multiple devices from different locations and shared on social media. Several different recordings could be fou…
Audio Conditioning for Music Generation via Discrete Bottleneck Features
Simon Rouard, Yossi Adi, Jade Copet +2
While most music generation models use textual or parametric conditioning (e.g. tempo, harmony, musical genre), we propose to condition a language model based music generation syst…
A Language Modeling Approach to Diacritic-Free Hebrew TTS
Amit Roth, Arnon Turetzky, Yossi Adi
We tackle the task of text-to-speech (TTS) in Hebrew. Traditional Hebrew contains Diacritics, which dictate the way individuals should pronounce given words, however, modern Hebrew…
HebDB: a Weakly Supervised Dataset for Hebrew Speech Processing
Arnon Turetzky, Or Tal, Yael Segal-Feldman +9
We present HebDB, a weakly supervised dataset for spoken language processing in the Hebrew language. HebDB offers roughly 2500 hours of natural and spontaneous speech recordings in…
Discrete Flow Matching
Itai Gat, Tal Remez, Neta Shaul +5
Despite Flow Matching and diffusion models having emerged as powerful generative paradigms for continuous variables such as images and videos, their application to high-dimensional…
NAST: Noise Aware Speech Tokenization for Speech Language Models
Shoval Messica, Yossi Adi
Speech tokenization is the task of representing speech signals as a sequence of discrete units. Such representations can be later used for various downstream tasks including automa…
Joint Audio and Symbolic Conditioning for Temporally Controlled Text-to-Music Generation
Or Tal, Alon Ziv, Itai Gat +2
We present JASCO, a temporally controlled text-to-music generation model utilizing both symbolic and audio-based conditions. JASCO can generate high-quality music samples condition…
The Interspeech 2024 Challenge on Speech Processing Using Discrete Units
Xuankai Chang, Jiatong Shi, Jinchuan Tian +7
Representing speech and audio signals in discrete units has become a compelling alternative to traditional high-dimensional feature vectors. Numerous studies have highlighted the e…
An Independence-promoting Loss for Music Generation with Language Models
Jean-Marie Lemercier, Simon Rouard, Jade Copet +2
Music generation schemes using language modeling rely on a vocabulary of audio tokens, generally provided as codes in a discrete latent space learnt by an auto-encoder. Multi-stage…
LaMI: Augmenting Large Language Models via Late Multi-Image Fusion
Guy Yariv, Idan Schwartz, Yossi Adi +1
Commonsense reasoning often requires both textual and visual knowledge, yet Large Language Models (LLMs) trained solely on text lack visual grounding (e.g., "what color is an emper…
The Larger the Better? Improved LLM Code-Generation via Budget Reallocation
Michael Hassid, Tal Remez, Jonas Gehring +2
It is a common belief that large language models (LLMs) are better than smaller-sized ones. However, larger models also require significantly more time and compute during inference…
Transformers are Multi-State RNNs
Matanel Oren, Michael Hassid, Nir Yarden +2
Transformers are considered conceptually different from the previous generation of state-of-the-art NLP models - recurrent neural networks (RNNs). In this work, we demonstrate that…
Masked Audio Generation using a Single Non-Autoregressive Transformer
Alon Ziv, Itai Gat, Gael Le Lan +6
We introduce MAGNeT, a masked generative sequence modeling method that operates directly over several streams of audio tokens. Unlike prior work, MAGNeT is comprised of a single-st…