271 citations · 460 across the 22 of their papers we have counts for
22 papers
A Knowledge-enhanced Two-stage Generative Framework for Medical Dialogue Information Extraction
Zefa Hu, Ziyi Ni, Jing Shi +2
This paper focuses on term-status pair extraction from medical dialogues (MD-TSPE), which is essential in diagnosis dialogue systems and the automatic scribe of electronic medical…
X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages
Feilong Chen, Minglun Han, Haozhi Zhao +4
Large language models (LLMs) have demonstrated remarkable language abilities. GPT-4, based on advanced LLMs, exhibits extraordinary multimodal capabilities beyond previous visual l…
Mixture of personality improved Spiking actor network for efficient multi-agent cooperation
Xiyun Li, Ziyi Ni, Jingqing Ruan +4
Adaptive human-agent and agent-agent cooperation are becoming more and more critical in the research area of multi-agent reinforcement learning (MARL), where remarked progress has…
VILAS: Exploring the Effects of Vision and Language Context in Automatic Speech Recognition
Ziyi Ni, Minglun Han, Feilong Chen +4
Enhancing automatic speech recognition (ASR) performance by leveraging additional multimodal information has shown promising results in previous studies. However, most of these wor…
Matching-based Term Semantics Pre-training for Spoken Patient Query Understanding
Zefa Hu, Xiuyi Chen, Haoran Wu +5
Medical Slot Filling (MSF) task aims to convert medical queries into structured information, playing an essential role in diagnosis dialogue systems. However, the lack of sufficien…
Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation
Minglun Han, Feilong Chen, Jing Shi +2
Large-scale pre-trained language models (PLMs) have shown great potential in natural language processing tasks. Leveraging the capabilities of PLMs to enhance automatic speech reco…
VLP: A Survey on Vision-Language Pre-training
Feilong Chen, Duzhen Zhang, Minglun Han +4
In the past few years, the emergence of pre-training models has brought uni-modal fields such as computer vision (CV) and natural language processing (NLP) to a new era. Substantia…
Discretization and Re-synthesis: an alternative method to solve the Cocktail Party Problem
Jing Shi, Xuankai Chang, Tomoki Hayashi +3
Deep learning based models have significantly improved the performance of speech separation with input mixtures like the cocktail party. Prominent methods (e.g., frequency-domain a…
Closing the Gap Between Time-Domain Multi-Channel Speech Enhancement on Real and Simulation Conditions
Wangyou Zhang, Jing Shi, Chenda Li +2
The deep learning based time-domain models, e.g. Conv-TasNet, have shown great potential in both single-channel and multi-channel speech enhancement. However, many experiments on t…
An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition
Xuankai Chang, Takashi Maekaku, Pengcheng Guo +8
Self-supervised pretraining on speech data has achieved a lot of progress. High-fidelity representation of the speech signal is learned from a lot of untranscribed data and shows p…
The 2020 ESPnet update: new features, broadened applications, performance improvements, and future plans
Shinji Watanabe, Florian Boyer, Xuankai Chang +12
This paper describes the recent development of ESPnet (https://github.com/espnet/espnet), an end-to-end speech processing toolkit. This project was initiated in December 2017 to ma…
Audio-visual Speech Separation with Adversarially Disentangled Visual Representation
Peng Zhang, Jiaming Xu, Jing shi +2
Speech separation aims to separate individual voice from an audio mixture of multiple simultaneous talkers. Although audio-only approaches achieve satisfactory performance, they bu…
ESPnet-se: end-to-end speech enhancement and separation toolkit designed for asr integration
Chenda Li, Jing Shi, Wangyou Zhang +8
We present ESPnet-SE, which is designed for the quick development of speech enhancement and speech separation systems in a single framework, along with the optional downstream spee…
Recent Developments on ESPnet Toolkit Boosted by Conformer
Pengcheng Guo, Florian Boyer, Xuankai Chang +12
In this study, we present recent developments on ESPnet: End-to-End Speech Processing toolkit, which mainly involves a recently proposed architecture called Conformer, Convolution-…
Training Noisy Single-Channel Speech Separation With Noisy Oracle Sources: A Large Gap and A Small Step
Matthew Maciejewski, Jing Shi, Shinji Watanabe +1
As the performance of single-channel speech separation systems has improved, there has been a desire to move to more challenging conditions than the clean, near-field speech that i…
Multi-task Metric Learning for Text-independent Speaker Verification
Yafeng Chen, Wu Guo, Jingjing Shi +2
In this work, we introduce metric learning (ML) to enhance the deep embedding learning for text-independent speaker verification (SV). Specifically, the deep speaker embedding netw…
Exploring Universal Speech Attributes for Speaker Verification with an Improved Cross-stitch Network
Jiajun Qi, Wu Guo, Jingjing Shi +2
The universal speech attributes for x-vector based speaker verification (SV) are addressed in this paper. The manner and place of articulation form the fundamental speech attribute…
Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signals
Jing Shi, Xuankai Chang, Pengcheng Guo +5
Neural sequence-to-sequence models are well established for applications which can be cast as mapping a single input sequence into a single output sequence. In this work, we focus…
Speaker-Conditional Chain Model for Speech Separation and Extraction
Jing Shi, Jiaming Xu, Yusuke Fujita +2
Speech separation has been extensively explored to tackle the cocktail party problem. However, these studies are still far from having enough generalization capabilities for real s…
Neural Speaker Diarization with Speaker-Wise Chain Rule
Yusuke Fujita, Shinji Watanabe, Shota Horiguchi +3
Speaker diarization is an essential step for processing multi-speaker audio. Although an end-to-end neural diarization (EEND) method achieved state-of-the-art performance, it is li…
Concept Learning through Deep Reinforcement Learning with Memory-Augmented Neural Networks
Jing Shi, Jiaming Xu, Yiqun Yao +1
Deep neural networks have shown superior performance in many regimes to remember familiar patterns with large amounts of data. However, the standard supervised deep learning paradi…
Hierarchical Memory Networks for Answer Selection on Unknown Words
Jiaming Xu, Jing Shi, Yiqun Yao +2
Recently, end-to-end memory networks have shown promising results on Question Answering task, which encode the past facts into an explicit memory and perform reasoning ability by m…