3.7k citations · 12.4k across the 59 of their papers we have counts for
5 papers · 1 filter
Monotonic Chunkwise Attention
Chung-Cheng Chiu, Colin Raffel
Sequence-to-sequence models with soft attention have been successfully applied to a wide variety of problems, but their decoding process incurs a quadratic time and space cost and…
Onsets and Frames: Dual-Objective Piano Transcription
Curtis Hawthorne, Erich Elsen, Jialin Song +6
We advance the state of the art in polyphonic piano music transcription by using a deep convolutional and recurrent neural network which is trained to jointly predict onsets and fr…
Learning Hard Alignments with Variational Inference
Dieterich Lawson, Chung-Cheng Chiu, George Tucker +3
There has recently been significant interest in hard attention models for tasks such as object recognition, visual captioning and speech recognition. Hard attention can offer benef…
Online and Linear-Time Attention by Enforcing Monotonic Alignments
Colin Raffel, Minh-Thang Luong, Peter J. Liu +2
Recurrent neural network models with an attention mechanism have proven to be extremely effective on a wide variety of sequence-to-sequence problems. However, the fact that soft at…
Training a Subsampling Mechanism in Expectation
Colin Raffel, Dieterich Lawson
We describe a mechanism for subsampling sequences and show how to compute its expected output so that it can be trained with standard backpropagation. We test this approach on a si…