An Asynchronous WFST-Based Decoder For Automatic Speech Recognition

Hang Lv, Zhehuai Chen, Hainan Xu +3

Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages

Yu Zhang, Wei Han, James Qin +24

Using Text Injection to Improve Recognition of Personal Identifiers in Speech

Yochai Blau, Rohan Agrawal, Lior Madmony +7

SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Model

Ke Hu, Ehsan Hosseini-Asl, Chen Chen +7

SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models

Chih-Kai Yang, Yen-Ting Piao, Tzu-Wen Hsu +8

Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence

NVIDIA, :, Amala Sanjay Deshmukh +204

How Auditory Knowledge in LLM Backbones Shapes Audio Language Models: A Holistic Evaluation

Ke-Han Lu, Szu-Wei Fu, Chao-Han Huck Yang +13

Injecting Text in Self-Supervised Speech Pretraining

Zhehuai Chen, Yu Zhang, Andrew Rosenberg +3

SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation

Zhehuai Chen, He Huang, Andrei Andrusenko +6

BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5

Zhehuai Chen, He Huang, Oleksii Hrinchuk +5

Instruction Data Generation and Unsupervised Adaptation for Speech Language Models

Vahid Noroozi, Zhehuai Chen, Somshubra Majumdar +3

Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition

Chao-Han Huck Yang, Taejin Park, Yuan Gong +18

Towards General Auditory Intelligence: Large Multimodal Models for Machine Listening and Speaking

Siyin Wang, Zengrui Jin, Changli Tang +26

MAESTRO: Matched Speech Text Representations through Modality Matching

Zhehuai Chen, Yu Zhang, Andrew Rosenberg +4

JOIST: A Joint Speech and Text Streaming Model For ASR

Tara N. Sainath, Rohit Prabhavalkar, Ankur Bapna +6

On Modular Training of Neural Acoustics-to-Word Model for LVCSR

Zhehuai Chen, Qi Liu, Hao Li +1

High-precision Voice Search Query Correction via Retrievable Speech-text Embedings

Christopher Li, Gary Wang, Kyle Kastner +9

Maestro-U: Leveraging joint speech-text representation learning for zero supervised speech ASR

Zhehuai Chen, Ankur Bapna, Andrew Rosenberg +4

Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning

Yuxin Li, Donghang Wu, Guan-Ting Lin +4

SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models

Zhen Wan, Chao-Han Huck Yang, Yahan Yu +8

VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning

Yifan Peng, Krishna C. Puvvada, Zhehuai Chen +7

Accented Speech Recognition: Benchmarking, Pre-training, and Diverse Data

Alëna Aksënova, Zhehuai Chen, Chung-Cheng Chiu +8

Linguistic Search Optimization for Deep Learning Based LVCSR

Zhehuai Chen

Audio Large Language Models Can Be Descriptive Speech Quality Evaluators

Chen Chen, Yuchen Hu, Siyin Wang +5

GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators

Yuchen Hu, Chen Chen, Chao-Han Huck Yang +4

Transducers with Pronunciation-aware Embeddings for Automatic Speech Recognition

Hainan Xu, Zhehuai Chen, Fei Jia +1

Voice Memory for Agentic Speech Recognition

Chao-Han Huck Yang, Zih-Ching Chen, Piotr Zelasko +3

The paper introduces Voice Memory, an inference-only framework for agentic speech recognition that uses a frozen corrector and a per‑domain memory file to decide when to modify hyp…

Understanding Shared Speech-Text Representations

Gary Wang, Kyle Kastner, Ankur Bapna +4

DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment

Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu +25

DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data

Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu +5

Anticipating Future with Large Language Model for Simultaneous Machine Translation

Siqi Ouyang, Oleksii Hrinchuk, Zhehuai Chen +4

Accelerating RNN-T Training and Inference Using CTC guidance

Yongqiang Wang, Zhehuai Chen, Chengjian Zheng +3

Sequence Discriminative Training for Deep Learning based Acoustic Keyword Spotting

Zhehuai Chen, Yanmin Qian, Kai Yu

End-to-end contextual speech recognition using class language models and a token passing decoder

Zhehuai Chen, Mahaveer Jain, Yongqiang Wang +2

EMMeTT: Efficient Multimodal Machine Translation Training

Piotr Żelasko, Zhehuai Chen, Mengru Wang +7

DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment

Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu +4

Modular End-to-end Automatic Speech Recognition Framework for Acoustic-to-word Model

Qi Liu, Zhehuai Chen, Hao Li +3

Progressive Joint Modeling in Unsupervised Single-channel Overlapped Speech Recognition

Zhehuai Chen, Jasha Droppo, Jinyu Li +1

Speech-Hands: A Self-Reflection Voice Agentic Approach to Speech Recognition and Audio Reasoning with Omni Perception

Zhen Wan, Chao-Han Huck Yang, Jinchuan Tian +15

Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks

Chien-yu Huang, Wei-Chih Chen, Shu-wen Yang +77

Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variations

Bo-Han Feng, Chien-Feng Liu, Yu-Hsuan Li Liang +9

Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens

Pin-Jui Ku, He Huang, Jean-Marie Lemercier +3

A GPU-based WFST Decoder with Exact Lattice Generation

Zhehuai Chen, Justin Luitjens, Hainan Xu +3

Less is More: Accurate Speech Recognition & Translation without Web-Scale Data

Krishna C. Puvvada, Piotr Żelasko, He Huang +9

Just A Rather Very Intelligent Spoken Agent

Chen Chen, Zhehuai Chen

Unsupervised Data Selection via Discrete Speech Representation for ASR

Zhiyun Lu, Yongqiang Wang, Yu Zhang +3

Chain-of-Thought Prompting for Speech Translation

Ke Hu, Zhehuai Chen, Chao-Han Huck Yang +5

Detecting the Undetectable: Assessing the Efficacy of Current Spoof Detection Methods Against Seamless Speech Edits

Sung-Feng Huang, Heng-Cheng Kuo, Zhehuai Chen +6

Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency

Guan-Ting Lin, Chen Chen, Zhehuai Chen +1

NeKo: Cross-Modality Post-Recognition Error Correction with Tasks-Guided Mixture-of-Experts Language Model

Yen-Ting Lin, Zhehuai Chen, Piotr Zelasko +11

Virtuoso: Massive Multilingual Speech-Text Joint Semi-Supervised Learning for Text-To-Speech

Takaaki Saeki, Heiga Zen, Zhehuai Chen +6

Word Level Timestamp Generation for Automatic Speech Recognition and Translation

Ke Hu, Krishna Puvvada, Elena Rastorgueva +7