4 citations · 8 across the 6 of their papers we have counts for
6 papers
Cross-Attention is all you need: Real-Time Streaming Transformers for Personalised Speech Enhancement
Shucong Zhang, Malcolm Chadwick, Alberto Gil C. P. Ramos +1
Personalised speech enhancement (PSE), which extracts only the speech of a target user and removes everything else from a recorded audio clip, can potentially improve users' experi…
Transformer-based Streaming ASR with Cumulative Attention
Mohan Li, Shucong Zhang, Catalin Zorila +1
In this paper, we propose an online attention mechanism, known as cumulative attention (CA), for streaming Transformer-based automatic speech recognition (ASR). Inspired by monoton…
Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers
Shucong Zhang, Cong-Thanh Do, Rama Doddipatla +3
Although the lower layers of a deep neural network learn features which are transferable across datasets, these layers are not transferable within the same dataset. That is, in gen…
On the Usefulness of Self-Attention for Automatic Speech Recognition with Transformers
Shucong Zhang, Erfan Loweimi, Peter Bell +1
Self-attention models such as Transformers, which can capture temporal relationships without being limited by the distance between events, have given competitive speech recognition…
Stochastic Attention Head Removal: A simple and effective method for improving Transformer Based ASR Models
Shucong Zhang, Erfan Loweimi, Peter Bell +1
Recently, Transformer based models have shown competitive automatic speech recognition (ASR) performance. One key factor in the success of these models is the multi-head attention…
When Can Self-Attention Be Replaced by Feed Forward Layers?
Shucong Zhang, Erfan Loweimi, Peter Bell +1
Recently, self-attention models such as Transformers have given competitive results compared to recurrent neural network systems in speech recognition. The key factor for the outst…