1 citations · 1 across the 9 of their papers we have counts for
9 papers
NormToken: Speaker- and Duration-Normalized Semantic Speech Tokens via Iterative S2U-T2U Refinement
Hanlin Zhang, Daxin Tan, Dehua Tao +3
Semantic speech tokens should preserve linguistic content while suppressing utterance-specific acoustic and duration variation. However, existing speech-to-unit (S2U) tokenizers of…
SpeechEditBench: A Bilingual Multi-Attribute Benchmark for Instruction-Guided Speech Editing
Hanlin Zhang, Daxin Tan, Dehua Tao +3
Instruction-guided speech editing requires a model to modify specified speech attributes while preserving non-target characteristics. Despite rapid progress in Speech Large Languag…
Speech-Omni-Lite: Portable Speech Interfaces for Vision-Language Models
Dehua Tao, Xuan Luo, Daxin Tan +5
While large-scale omni-models have demonstrated impressive capabilities across various modalities, their strong performance heavily relies on massive multimodal data and incurs sub…
PROST-LLM: Progressively Enhancing the Speech-to-Speech Translation Capability in LLMs
Jing Xu, Jiaqi Wang, Daxin Tan +1
Although Large Language Models (LLMs) excel in many tasks, their application to Speech-to-Speech Translation (S2ST) is underexplored and hindered by data scarcity. To bridge this g…
DSA-Tokenizer: Disentangled Semantic-Acoustic Tokenization via Flow Matching-based Hierarchical Fusion
Hanlin Zhang, Daxin Tan, Dehua Tao +5
Speech tokenizers are a key building block of fully discrete Speech LLMs. Existing tokenizers either prioritize semantic encoding, fuse semantic content with acoustic style insepar…
UPME: An Unsupervised Peer Review Framework for Multimodal Large Language Model Evaluation
Qihui Zhang, Munan Ning, Zheyuan Liu +7
Multimodal Large Language Models (MLLMs) have emerged to tackle the challenges of Visual Question Answering (VQA), sparking a new research focus on conducting objective evaluations…
Exploring SSL Discrete Tokens for Multilingual ASR
Mingyu Cui, Daxin Tan, Yifan Yang +5
With the advancement of Self-supervised Learning (SSL) in speech-related tasks, there has been growing interest in utilizing discrete tokens generated by SSL for automatic speech r…
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions
Kai Chen, Yunhao Gou, Runhui Huang +28
GPT-4o, an omni-modal model that enables vocal conversations with diverse emotions and tones, marks a milestone for omni-modal foundation models. However, empowering Large Language…
Enhancing Code-switched Text-to-Speech Synthesis Capability in Large Language Models with only Monolingual Corpora
Jing Xu, Daxin Tan, Jiaqi Wang +1
While Large Language Models (LLMs) have shown potential in speech generation and recognition, their applications are mainly confined to monolingual scenarios, with limited explorat…