Hybrid Neural Models For Sequence Modelling: The Best Of Three Worlds
arXiv:1909.07102
Abstract
We propose a neural architecture with the main characteristics of the most successful neural models of the last years: bidirectional RNNs, encoder-decoder, and the Transformer model. Evaluation on three sequence labelling tasks yields results that are close to the state-of-the-art for all tasks and better than it for some of them, showing the pertinence of this hybrid architecture for this kind of tasks.
arXiv comment12 pages + bibliography. English version, and slight improvement, of the paper published at the French conference TALN 2019
References in corpus (6)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Sequence to Sequence Learning with Neural Networks
- Natural Language Processing (almost) from Scratch
- Universal Transformers
- Improving Recurrent Neural Networks For Sequence Labelling
- Seq2Biseq: Bidirectional Output-wise Recurrent Neural Networks for Sequence Modelling