Sequence Prediction with Neural Segmental Models
arXiv:1709.01572
Abstract
Segments that span contiguous parts of inputs, such as phonemes in speech, named entities in sentences, actions in videos, occur frequently in sequence prediction problems. Segmental models, a class of models that explicitly hypothesizes segments, have allowed the exploration of rich segment features for sequence prediction. However, segmental models suffer from slow decoding, hampering the use of computationally expensive features. In this thesis, we introduce discriminative segmental cascades, a multi-pass inference framework that allows us to improve accuracy by adding higher-order features and neural segmental features while maintaining efficiency. We also show that instead of including more features to obtain better accuracy, segmental cascades can be used to speed up training and decoding. Segmental models, similarly to conventional speech recognizers, are typically trained in multiple stages. In the first stage, a frame classifier is trained with manual alignments, and then in the second stage, segmental models are trained with manual alignments and the out- puts of the frame classifier. However, obtaining manual alignments are time-consuming and expensive. We explore end-to-end training for segmental models with various loss functions, and show how end-to-end training with marginal log loss can eliminate the need for detailed manual alignments. We draw the connections between the marginal log loss and a popular end-to-end training approach called connectionist temporal classification. We present a unifying framework for various end-to-end graph search-based models, such as hidden Markov models, connectionist temporal classification, and segmental models. Finally, we discuss possible extensions of segmental models to large-vocabulary sequence prediction tasks.
arXiv commentThesis
References in corpus (17)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Do Deep Nets Really Need to be Deep?
- Sequence Transduction with Recurrent Neural Networks
- Deep Scattering Spectrum
- Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
- Weighted Automata in Text and Speech Processing
- Variants of RMSProp and Adagrad with Logarithmic Regret Bounds
- Segmental Recurrent Neural Networks
- A segmental framework for fully-unsupervised large-vocabulary speech recognition
- Advances in All-Neural Speech Recognition
- Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition
- Segmental Recurrent Neural Networks for End-to-end Speech Recognition
- Discriminative Segmental Cascades for Feature-Rich Phone Recognition
- Multitask Learning with CTC and Segmental CRF for Speech Recognition
- Unsupervised Neural Hidden Markov Models
- Risk Minimization in Structured Prediction using Orbit Loss
- End-to-End Training Approaches for Discriminative Segmental Models