310 citations · 498 across the 18 of their papers we have counts for
4 papers · 1 filter
Lattice-Free MMI Adaptation Of Self-Supervised Pretrained Acoustic Models
Apoorv Vyas, Srikanth Madikeri, Hervé Bourlard
In this work, we propose lattice-free MMI (LFMMI) for supervised adaptation of self-supervised pretrained acoustic model. We pretrain a Transformer model on thousand hours of untra…
Pkwrap: a PyTorch Package for LF-MMI Training of Acoustic Models
Srikanth Madikeri, Sibo Tong, Juan Zuluaga-Gomez +3
We present a simple wrapper that is useful to train acoustic models in PyTorch using Kaldi's LF-MMI training framework. The wrapper, called pkwrap (short form of PyTorch kaldi wrap…
Fast Transformers with Clustered Attention
Apoorv Vyas, Angelos Katharopoulos, François Fleuret
Transformers have been proven a successful model for a variety of tasks in sequence modeling. However, computing the attention matrix, which is their key component, has quadratic c…
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas +1
Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long se…