6 citations · 16 across the 19 of their papers we have counts for
13 papers · 1 filter
Teaching LLMs to Hear Who Spoke What: Metadata-Supervised Pretraining for Encoder-Free Speech-LLMs
Mohan Shi, Ruchao Fan, Sunit Sivasankaran +2
Encoder-based speech large language models (Speech-LLMs) commonly employ pretrained speech encoders that prioritize linguistic content but may discard fine-grained acoustic cues es…
Compress the Cache, Not the Speech Embedding: KV Compression for Efficient Speech LLMs
Ke-Han Lu, Keqi Deng, Ruchao Fan +2
Speech large language models (Speech LLMs) typically encode speech into sequences far longer than text, creating a major efficiency bottleneck during autoregressive decoding. A com…
OpenSTBench: Beyond Semantic Evaluation for Speech Translation
Yanjie An, Yuxiang Zhao, Yichi Zhang +5
Speech translation systems increasingly span speech-to-text translation (S2TT), speech-to-speech translation (S2ST), offline translation, and streaming generation, producing output…
Wav2Prompt: End-to-End Speech Prompt Generation and Tuning For LLM in Zero and Few-shot Learning
Keqi Deng, Guangzhi Sun, Philip C. Woodland
Wav2Prompt is proposed which allows straightforward integration between spoken input and a text-based large language model (LLM). Wav2Prompt uses a simple training process with onl…
FastInject: Injecting Unpaired Text Data into CTC-based ASR training
Keqi Deng, Philip C. Woodland
Recently, connectionist temporal classification (CTC)-based end-to-end (E2E) automatic speech recognition (ASR) models have achieved impressive results, especially with the develop…
Label-Synchronous Neural Transducer for Adaptable Online E2E Speech Recognition
Keqi Deng, Philip C. Woodland
Although end-to-end (E2E) automatic speech recognition (ASR) has shown state-of-the-art recognition accuracy, it tends to be implicitly biased towards the training data distributio…
Decoupled Structure for Improved Adaptability of End-to-End Models
Keqi Deng, Philip C. Woodland
Although end-to-end (E2E) trainable automatic speech recognition (ASR) has shown great success by jointly learning acoustic and linguistic information, it still suffers from the ef…
Label-Synchronous Neural Transducer for End-to-End ASR
Keqi Deng, Philip C. Woodland
Neural transducers provide a natural way of streaming ASR. However, they augment output sequences with blank tokens which leads to challenges for domain adaptation using text data.…
Adaptable End-to-End ASR Models using Replaceable Internal LMs and Residual Softmax
Keqi Deng, Philip C. Woodland
End-to-end (E2E) automatic speech recognition (ASR) implicitly learns the token sequence distribution of paired audio-transcript training data. However, it still suffers from domai…
Improving Streaming End-to-End ASR on Transformer-based Causal Models with Encoder States Revision Strategies
Zehan Li, Haoran Miao, Keqi Deng +4
There is often a trade-off between performance and latency in streaming automatic speech recognition (ASR). Traditional methods such as look-ahead and chunk-based methods, usually…
Improving non-autoregressive end-to-end speech recognition with pre-trained acoustic and language models
Keqi Deng, Zehui Yang, Shinji Watanabe +3
While Transformers have achieved promising results in end-to-end (E2E) automatic speech recognition (ASR), their autoregressive (AR) structure becomes a bottleneck for speeding up…
Improving Hybrid CTC/Attention End-to-end Speech Recognition with Pretrained Acoustic and Language Model
Keqi Deng, Songjun Cao, Yike Zhang +1
Recently, self-supervised pretraining has achieved impressive results in end-to-end (E2E) automatic speech recognition (ASR). However, the dominant sequence-to-sequence (S2S) E2E m…
Improving Accent Identification and Accented Speech Recognition Under a Framework of Self-supervised Learning
Keqi Deng, Songjun Cao, Long Ma
Recently, self-supervised pre-training has gained success in automatic speech recognition (ASR). However, considering the difference between speech accents in real scenarios, how t…