13 citations · 26 across the 21 of their papers we have counts for
21 papers
STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelity
Sitong Cheng, Weizhen Bian, Songjun Cao +9
Speech-to-speech translation (S2ST) should preserve not only lexical meaning, but also expressive attributes: emotion, scenario style (e.g., news reporting vs. dramatic dialogue),…
MathVis-Fine: Aligning Visual Supervision with Necessity via Progressive Dependency-Guided Training for Multimodal Mathematical Reasoning
Wanshi Xu, Haokun Zhao, Haidong Yuan +2
Chain-of-Thought (CoT) reasoning has extended from purely linguistic domains to multimodal scenarios; however, existing approaches often treat visual inputs as homogeneous or auxil…
Diffusion Reconstruction towards Generalizable Audio Deepfake Detection
Bo Cheng, Songjun Cao, Xiaoming Zhang +3
Achieving robust generalization against unseen attacks remains a challenge in Audio Deepfake Detection (ADD), driven by the rapid evolution of generative models. To address this, w…
Controllable Spoken Dialogue Generation: An LLM-Driven Grading System for K-12 Non-Native English Learners
Haidong Yuan, Haokun Zhao, Wanshi Xu +4
Large language models (LLMs) often fail to meet the pedagogical needs of K-12 English learners in non-native contexts due to a proficiency mismatch. To address this widespread chal…
Thinking with Constructions: A Benchmark and Policy Optimization for Visual-Text Interleaved Geometric Reasoning
Haokun Zhao, Wanshi Xu, Haidong Yuan +3
Geometric reasoning inherently requires "thinking with constructions" -- the dynamic manipulation of visual aids to bridge the gap between problem conditions and solutions. However…
Leveraging large multimodal models for audio-video deepfake detection: a pilot study
Songjun Cao, Yuqi Li, Yunpeng Luo +2
Audio-visual deepfake detection (AVD) is increasingly important as modern generators can fabricate convincing speech and video. Most current multimodal detectors are small, task-sp…
PseudoVC: Improving One-shot Voice Conversion with Pseudo Paired Data
Songjun Cao, Qinghua Wu, Jie Chen +2
As parallel training data is scarce for one-shot voice conversion (VC) tasks, waveform reconstruction is typically performed by various VC systems. A typical one-shot VC system com…
MPE-TTS: Customized Emotion Zero-Shot Text-To-Speech Using Multi-Modal Prompt
Zhichao Wu, Yueteng Kang, Songjun Cao +3
Most existing Zero-Shot Text-To-Speech(ZS-TTS) systems generate the unseen speech based on single prompt, such as reference speech or text descriptions, which limits their flexibil…
Detect All-Type Deepfake Audio: Wavelet Prompt Tuning for Enhanced Auditory Perception
Yuankun Xie, Ruibo Fu, Zhiyong Wang +5
The rapid advancement of audio generation technologies has escalated the risks of malicious deepfake audio across speech, sound, singing voice, and music, threatening multimedia se…
DiffCSS: Diverse and Expressive Conversational Speech Synthesis with Diffusion Models
Weihao wu, Zhiwei Lin, Yixuan Zhou +6
Conversational speech synthesis (CSS) aims to synthesize both contextually appropriate and expressive speech, and considerable efforts have been made to enhance the understanding o…
Neural Codec Source Tracing: Toward Comprehensive Attribution in Open-Set Condition
Yuankun Xie, Xiaopeng Wang, Zhiyong Wang +7
Current research in audio deepfake detection is gradually transitioning from binary classification to multi-class tasks, referred as audio deepfake source tracing task. However, ex…
A Transcription Prompt-based Efficient Audio Large Language Model for Robust Speech Recognition
Yangze Li, Xiong Wang, Songjun Cao +3
Audio-LLM introduces audio modality into a large language model (LLM) to enable a powerful LLM to recognize, understand, and generate audio. However, during speech recognition in n…
DistillW2V2: A Small and Streaming Wav2vec 2.0 Based ASR Model
Yanzhe Fu, Yueteng Kang, Songjun Cao +1
Wav2vec 2.0 (W2V2) has shown impressive performance in automatic speech recognition (ASR). However, the large model size and the non-streaming architecture make it hard to be used…
Censer: Curriculum Semi-supervised Learning for Speech Recognition Based on Self-supervised Pre-training
Bowen Zhang, Songjun Cao, Xiaoming Zhang +3
Recent studies have shown that the benefits provided by self-supervised pre-training and self-training (pseudo-labeling) are complementary. Semi-supervised fine-tuning strategies u…
A practical framework for multi-domain speech recognition and an instance sampling method to neural language modeling
Yike Zhang, Xiaobing Feng, Yi Liu +2
Automatic speech recognition (ASR) systems used on smart phones or vehicles are usually required to process speech queries from very different domains. In such situations, a vanill…
Improving CTC-based speech recognition via knowledge transferring from pre-trained language models
Keqi Deng, Songjun Cao, Yike Zhang +4
Recently, end-to-end automatic speech recognition models based on connectionist temporal classification (CTC) have achieved impressive results, especially when fine-tuned from wav2…
Improving Hybrid CTC/Attention End-to-end Speech Recognition with Pretrained Acoustic and Language Model
Keqi Deng, Songjun Cao, Yike Zhang +1
Recently, self-supervised pretraining has achieved impressive results in end-to-end (E2E) automatic speech recognition (ASR). However, the dominant sequence-to-sequence (S2S) E2E m…
Improving Accent Identification and Accented Speech Recognition Under a Framework of Self-supervised Learning
Keqi Deng, Songjun Cao, Long Ma
Recently, self-supervised pre-training has gained success in automatic speech recognition (ASR). However, considering the difference between speech accents in real scenarios, how t…
Improving Streaming Transformer Based ASR Under a Framework of Self-supervised Learning
Songjun Cao, Yueteng Kang, Yanzhe Fu +4
Recently self-supervised learning has emerged as an effective approach to improve the performance of automatic speech recognition (ASR). Under such a framework, the neural network…
Improving Speech Recognition Accuracy of Local POI Using Geographical Models
Songjun Cao, Yike Zhang, Xiaobing Feng +1
Nowadays voice search for points of interest (POI) is becoming increasingly popular. However, speech recognition for local POI has remained to be a challenge due to multi-dialect a…
Multi-head Monotonic Chunkwise Attention For Online Speech Recognition
Baiji Liu, Songjun Cao, Sining Sun +2
The attention mechanism of the Listen, Attend and Spell (LAS) model requires the whole input sequence to calculate the attention context and thus is not suitable for online speech…