5 papers
A Treatise On FST Lattice Based MMI Training
Adnan Haider, Tim Ng, Zhen Huang +2
Maximum mutual information (MMI) has become one of the two de facto methods for sequence-level training of speech recognition acoustic models. This paper aims to isolate, identify…
A Distributed Optimisation Framework Combining Natural Gradient with Hessian-Free for Discriminative Sequence Training
Adnan Haider, Chao Zhang, Florian L. Kreyssig +1
This paper presents a novel natural gradient and Hessian-free (NGHF) optimisation framework for neural network training that can operate efficiently in a distributed manner. It rel…
Combining Natural Gradient with Hessian Free Methods for Sequence Training
Adnan Haider, P. C. Woodland
This paper presents a new optimisation approach to train Deep Neural Networks (DNNs) with discriminative sequence criteria. At each iteration, the method combines information from…
Sequence Training of DNN Acoustic Models With Natural Gradient
Adnan Haider, Philip C. Woodland
Deep Neural Network (DNN) acoustic models often use discriminative sequence training that optimises an objective function that better approximates the word error rate (WER) than fr…
A Common Framework for Natural Gradient and Taylor based Optimisation using Manifold Theory
Adnan Haider
This technical report constructs a theoretical framework to relate standard Taylor approximation based optimisation methods with Natural Gradient (NG), a method which is Fisher eff…