1 citations · 1 across the 11 of their papers we have counts for
12 papers
Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation
Daxin Tan, Dehua Tao, Chengxi Deng +2
Autoregressive generation of interleaved text and acoustic tokens is a common approach to spoken-response generation in speech large language models. Although this design enables s…
NormToken: Speaker- and Duration-Normalized Semantic Speech Tokens via Iterative S2U-T2U Refinement
Hanlin Zhang, Daxin Tan, Dehua Tao +3
Semantic speech tokens should preserve linguistic content while suppressing utterance-specific acoustic and duration variation. However, existing speech-to-unit (S2U) tokenizers of…
SpeechEditBench: A Bilingual Multi-Attribute Benchmark for Instruction-Guided Speech Editing
Hanlin Zhang, Daxin Tan, Dehua Tao +3
Instruction-guided speech editing requires a model to modify specified speech attributes while preserving non-target characteristics. Despite rapid progress in Speech Large Languag…
A Survey of Audio Reasoning in Multimodal Foundation Models
Zhihan Guo, Wenqian Cui, Guan-Ting Lin +8
Reasoning has become a defining capability of modern foundation models, yet its development in the audio modality remains limited. Audio poses challenges that are distinct from tho…
Minimizing Modality Gap from the Input Side: Your Speech LLM Can Be a Prosody-Aware Text LLM
Wenqian Cui, Xiao-Hui Li, Daxin Tan +2
Speech large language models (SLMs) are typically built from text large language model (TLM) checkpoints, yet they still suffer from a substantial modality gap. Prior work has main…
Speech-Omni-Lite: Portable Speech Interfaces for Vision-Language Models
Dehua Tao, Xuan Luo, Daxin Tan +5
While large-scale omni-models have demonstrated impressive capabilities across various modalities, their strong performance heavily relies on massive multimodal data and incurs sub…
PROST-LLM: Progressively Enhancing the Speech-to-Speech Translation Capability in LLMs
Jing Xu, Jiaqi Wang, Daxin Tan +1
Although Large Language Models (LLMs) excel in many tasks, their application to Speech-to-Speech Translation (S2ST) is underexplored and hindered by data scarcity. To bridge this g…
AEQ-Bench: Measuring Empathy of Omni-Modal Large Models
Xuan Luo, Lewei Yao, Libo Zhao +6
While the automatic evaluation of omni-modal large models (OLMs) is essential, assessing empathy remains a significant challenge due to its inherent affectivity. To investigate thi…
DSA-Tokenizer: Disentangled Semantic-Acoustic Tokenization via Flow Matching-based Hierarchical Fusion
Hanlin Zhang, Daxin Tan, Dehua Tao +5
Speech tokenizers are a key building block of fully discrete Speech LLMs. Existing tokenizers either prioritize semantic encoding, fuse semantic content with acoustic style insepar…
Exploring SSL Discrete Tokens for Multilingual ASR
Mingyu Cui, Daxin Tan, Yifan Yang +5
With the advancement of Self-supervised Learning (SSL) in speech-related tasks, there has been growing interest in utilizing discrete tokens generated by SSL for automatic speech r…
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions
Kai Chen, Yunhao Gou, Runhui Huang +28
GPT-4o, an omni-modal model that enables vocal conversations with diverse emotions and tones, marks a milestone for omni-modal foundation models. However, empowering Large Language…
Enhancing Code-switched Text-to-Speech Synthesis Capability in Large Language Models with only Monolingual Corpora
Jing Xu, Daxin Tan, Jiaqi Wang +1
While Large Language Models (LLMs) have shown potential in speech generation and recognition, their applications are mainly confined to monolingual scenarios, with limited explorat…