7 papers
Speaker Attributed Automatic Speech Recognition Using Speech Aware LLMS
Hagai Aronowitz, Zvi Kons, Avihu Dekel +2
Speaker-Attributed Automatic Speech Recognition (SAA) enhances traditional ASR systems by incorporating relative speaker identity tags directly into the transcript (e.g., [Speaker…
Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark
Arnon Turetzky, Avihu Dekel, Hagai Aronowitz +2
Spoken meaning often depends not only on what is said, but also on which word is emphasized. The same sentence can convey correction, contrast, or clarification depending on where…
Speech Synthesis From Continuous Features Using Per-Token Latent Diffusion
Arnon Turetzky, Avihu Dekel, Nimrod Shabtay +5
We present SALAD, a zero-shot TTS autoregressive model operating over continuous speech representations. SALAD utilizes a per-token diffusion process to refine and predict continuo…
Advancing Speech Understanding in Speech-Aware Language Models with GRPO
Avishai Elmakies, Hagai Aronowitz, Nimrod Shabtay +3
In this paper, we introduce a Group Relative Policy Optimization (GRPO)-based method for training Speech-Aware Large Language Models (SALLMs) on open-format speech understanding ta…
Spoken question answering for visual queries
Nimrod Shabtay, Zvi Kons, Avihu Dekel +3
Question answering (QA) systems are designed to answer natural language questions. Visual QA (VQA) and Spoken QA (SQA) systems extend the textual QA system to accept visual and spo…
Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities
George Saon, Avihu Dekel, Alexander Brooks +21
Granite-speech LLMs are compact and efficient speech language models specifically designed for English ASR and automatic speech translation (AST). The models were trained by modali…