27 citations · 60 across the 30 of their papers we have counts for
32 papers
Dual-Form ASR: Semantics-Aware Inverse Text Normalization for Chinese Speech Recognition
Fengrun Zhang, Li Fu, Wangjin Zhou +3
Modern automatic speech recognition (ASR) scenarios require both spoken-form transcripts for faithful transcription and readable written-form transcripts with inverse text normaliz…
JoyStreamer: Unlocking Highly Expressive Avatars via Harmonized Text-Audio Conditioning
Ruikui Wang, Jinheng Feng, Lang Tian +6
Existing video avatar models have demonstrated impressive capabilities in scenarios such as talking, public speaking, and singing. However, the majority of these methods exhibit li…
JoyStreamer-Flash: Real-time and Infinite Audio-Driven Avatar Generation with Autoregressive Diffusion
Chaochao Li, Ruikui Wang, Liangbo Zhou +5
Existing DiT-based audio-driven avatar generation methods have achieved considerable progress, yet their broader application is constrained by limitations such as high computationa…
PAC: Pronunciation-Aware Contextualized Large Language Model-based Automatic Speech Recognition
Li Fu, Yu Xin, Sunlu Zeng +3
This paper presents a Pronunciation-Aware Contextualized (PAC) framework to address two key challenges in Large Language Model (LLM)-based Automatic Speech Recognition (ASR) system…
UME: Upcycling Mixture-of-Experts for Scalable and Efficient Automatic Speech Recognition
Li Fu, Shanyong Yu, Siqi Li +3
Recent advancements in scaling up models have significantly improved performance in Automatic Speech Recognition (ASR) tasks. However, training large ASR models from scratch remain…
PAS: Data-Efficient Plug-and-Play Prompt Augmentation System
Miao Zheng, Hao Liang, Fan Yang +16
In recent years, the rise of Large Language Models (LLMs) has spurred a growing demand for plug-and-play AI systems. Among the various AI techniques, prompt engineering stands out…
Leveraging Label Information for Multimodal Emotion Recognition
Peiying Wang, Sunlu Zeng, Junqing Chen +4
Multimodal emotion recognition (MER) aims to detect the emotional status of a given expression by combining the speech and text information. Intuitively, label information should b…
AUGUST: an Automatic Generation Understudy for Synthesizing Conversational Recommendation Datasets
Yu Lu, Junwei Bao, Zichen Ma +4
High-quality data is essential for conversational recommendation systems and serves as the cornerstone of the network architecture development and training strategy design. Existin…
OTF: Optimal Transport based Fusion of Supervised and Self-Supervised Learning Models for Automatic Speech Recognition
Li Fu, Siqi Li, Qingtao Li +6
Self-Supervised Learning (SSL) Automatic Speech Recognition (ASR) models have shown great promise over Supervised Learning (SL) ones in low-resource settings. However, the advantag…
Multi-Speaker Multi-Style Speech Synthesis with Timbre and Style Disentanglement
Wei Song, Yanghao Yue, Ya-jie Zhang +3
Disentanglement of a speaker's timbre and style is very important for style transfer in multi-speaker multi-style text-to-speech (TTS) scenarios. With the disentanglement of timbre…
SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentation
Huaishao Luo, Junwei Bao, Youzheng Wu +2
Recently, the contrastive language-image pre-training, e.g., CLIP, has demonstrated promising results on various downstream tasks. The pre-trained model can capture enriched visual…
MaskedSpeech: Context-aware Speech Synthesis with Masking Strategy
Ya-Jie Zhang, Wei Song, Yanghao Yue +3
Humans often speak in a continuous manner which leads to coherent and consistent prosody properties across neighboring utterances. However, most state-of-the-art speech synthesis s…
MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Tracking
Haoning Zhang, Junwei Bao, Haipeng Sun +4
Dialogue state tracking (DST) aims to convert the dialogue history into dialogue states which consist of slot-value pairs. As condensed structural information memorizing all histor…
PLM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training
Junwei Bao, Yifan Wang, Jiangyong Ying +4
Conventional autoregressive left-to-right (L2R) sequence generation faces two issues during decoding: limited to unidirectional target sequence modeling, and constrained on strong…
MuGER: Multi-Granularity Evidence Retrieval and Reasoning for Hybrid Question Answering
Yingyao Wang, Junwei Bao, Chaoqun Duan +3
Hybrid question answering (HQA) aims to answer questions over heterogeneous data, including tables and passages linked to table cells. The heterogeneous data can provide different…
UniRPG: Unified Discrete Reasoning over Table and Text as Program Generation
Yongwei Zhou, Junwei Bao, Chaoqun Duan +3
Question answering requiring discrete reasoning, e.g., arithmetic computing, comparison, and counting, over knowledge is a challenging task. In this paper, we propose UniRPG, a sem…
Mars: Modeling Context & State Representations with Contrastive Learning for End-to-End Task-Oriented Dialog
Haipeng Sun, Junwei Bao, Youzheng Wu +1
Traditional end-to-end task-oriented dialog systems first convert dialog context into belief state and action state before generating the system response. The system response perfo…
AutoQGS: Auto-Prompt for Low-Resource Knowledge-based Question Generation from SPARQL
Guanming Xiong, Junwei Bao, Wen Zhao +2
This study investigates the task of knowledge-based question generation (KBQG). Conventional KBQG works generated questions from fact triples in the knowledge graph, which could no…
LUNA: Learning Slot-Turn Alignment for Dialogue State Tracking
Yifan Wang, Jing Zhao, Junwei Bao +3
Dialogue state tracking (DST) aims to predict the current dialogue state given the dialogue history. Existing methods generally exploit the utterances of all dialogue turns to assi…
BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialog
Haipeng Sun, Junwei Bao, Youzheng Wu +1
A typical end-to-end task-oriented dialog system transfers context into dialog state, and upon which generates a response, which usually faces the problem of error propagation from…
OPERA:Operation-Pivoted Discrete Reasoning over Text
Yongwei Zhou, Junwei Bao, Chaoqun Duan +7
Machine reading comprehension (MRC) that requires discrete reasoning involving symbolic operations, e.g., addition, sorting, and counting, is a challenging task. According to this…
Fine- and Coarse-Granularity Hybrid Self-Attention for Efficient BERT
Jing Zhao, Yifan Wang, Junwei Bao +2
Transformer-based pre-trained models, such as BERT, have shown extraordinary success in achieving state-of-the-art results in many natural language processing applications. However…
SCaLa: Supervised Contrastive Learning for End-to-End Speech Recognition
Li Fu, Xiaoxiao Li, Runyu Wang +5
End-to-end Automatic Speech Recognition (ASR) models are usually trained to optimize the loss of the whole token sequence, while neglecting explicit phonemic-granularity supervisio…
The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service
Nan Zhao, Haoran Li, Youzheng Wu +2
With the development of the Internet, more and more people get accustomed to online shopping. When communicating with customer service, users may express their requirements by mean…