33 citations · 49 across the 13 of their papers we have counts for
17 papers
Watch and Learn: Teaching Visual In-Context Learning by Removing Textual Shortcuts
Nimrod Shabtay, Asaf Yehudai, Avihu Dekel +2
Vision-language models (VLMs) struggle with in context learning compared to their textual counterparts. Our preliminary study shows that VLMs rely on textual shortcuts and underuti…
Design of the IBM Granite 5.0 TurboCTC ASR Model
Brian Kingsbury, George Saon, Masayuki Suzuki +5
We describe the architecture, training methodology and inference speedups of Granite 5.0 Turbo CTC, a 470 million parameter encoder-only model with an excellent speed-accuracy trad…
Word Timestamps and Speaker Attribution with a Non-Autoregressive LLM
Zvi Kons, Avihu Dekel, Hagai Aronowitz +2
Timestamps and speaker attribution are useful additions to speech recognition, creating a rich text transcript. This information can either be extracted during transcription or ali…
Speaker Attributed Automatic Speech Recognition Using Speech Aware LLMS
Hagai Aronowitz, Zvi Kons, Avihu Dekel +2
Speaker-Attributed Automatic Speech Recognition (SAA) enhances traditional ASR systems by incorporating relative speaker identity tags directly into the transcript (e.g., [Speaker…
Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark
Arnon Turetzky, Avihu Dekel, Hagai Aronowitz +2
Spoken meaning often depends not only on what is said, but also on which word is emphasized. The same sentence can convey correction, contrast, or clarification depending on where…
Balanced Thinking: Improving Chain of Thought Training in Vision Language Models
Shaked Perek, Ben Wiesel, Avihu Dekel +2
Multimodal reasoning in vision-language models (VLMs) typically relies on a two-stage process: supervised fine-tuning (SFT) and reinforcement learning (RL). In standard SFT, all to…
Self-Speculative Decoding for LLM-based ASR with CTC Encoder Drafts
George Saon, Samuel Thomas, Takashi Fukuda +3
We propose self-speculative decoding for speech-aware LLMs by using the CTC encoder as a draft model to accelerate auto-regressive (AR) inference and improve ASR accuracy. Our thre…
NLE: Non-autoregressive LLM-based ASR by Transcript Editing
Avihu Dekel, Samuel Thomas, Takashi Fukada +1
While autoregressive (AR) LLM-based ASR systems achieve strong accuracy, their sequential decoding limits parallelism and incurs high latency. We propose NLE, a non-autoregressive…
Advancing Speech Understanding in Speech-Aware Language Models with GRPO
Avishai Elmakies, Hagai Aronowitz, Nimrod Shabtay +3
In this paper, we introduce a Group Relative Policy Optimization (GRPO)-based method for training Speech-Aware Large Language Models (SALLMs) on open-format speech understanding ta…
Spoken question answering for visual queries
Nimrod Shabtay, Zvi Kons, Avihu Dekel +3
Question answering (QA) systems are designed to answer natural language questions. Visual QA (VQA) and Spoken QA (SQA) systems extend the textual QA system to accept visual and spo…
Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities
George Saon, Avihu Dekel, Alexander Brooks +21
Granite-speech LLMs are compact and efficient speech language models specifically designed for English ASR and automatic speech translation (AST). The models were trained by modali…
Low Bitrate High-Quality RVQGAN-based Discrete Speech Tokenizer
Slava Shechtman, Avihu Dekel
Discrete Audio codecs (or audio tokenizers) have recently regained interest due to the ability of Large Language Models (LLMs) to learn their compressed acoustic representations. V…
Speech Synthesis From Continuous Features Using Per-Token Latent Diffusion
Arnon Turetzky, Avihu Dekel, Nimrod Shabtay +5
We present SALAD, a zero-shot TTS autoregressive model operating over continuous speech representations. SALAD utilizes a per-token diffusion process to refine and predict continuo…
Exploring the Benefits of Tokenization of Discrete Acoustic Units
Avihu Dekel, Raul Fernandez
Tokenization algorithms that merge the units of a base vocabulary into larger, variable-rate units have become standard in natural language processing tasks. This idea, however, ha…
Speak While You Think: Streaming Speech Synthesis During Text Generation
Avihu Dekel, Slava Shechtman, Raul Fernandez +3
Large Language Models (LLMs) demonstrate impressive capabilities, yet interaction with these models is mostly facilitated through text. Using Text-To-Speech to synthesize LLM outpu…
Active Learning Through a Covering Lens
Ofer Yehuda, Avihu Dekel, Guy Hacohen +1
Deep active learning aims to reduce the annotation cost for the training of deep models, which is notoriously data-hungry. Until recently, deep active learning methods were ineffec…
Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets
Guy Hacohen, Avihu Dekel, Daphna Weinshall
Investigating active learning, we focus on the relation between the number of labeled examples (budget size), and suitable querying strategies. Our theoretical analysis shows a beh…