FastFusionNet: New State-of-the-Art for DAWNBench SQuAD
arXiv:1902.11291
Abstract
In this technical report, we introduce FastFusionNet, an efficient variant of FusionNet [12]. FusionNet is a high performing reading comprehension architecture, which was designed primarily for maximum retrieval accuracy with less regard towards computational requirements. For FastFusionNets we remove the expensive CoVe layers [21] and substitute the BiLSTMs with far more efficient SRU layers [19]. The resulting architecture obtains state-of-the-art results on DAWNBench [5] while achieving the lowest training and inference time on SQuAD [25] to-date. The code is available at https://github.com/felixgwu/FastFusionNet.
arXiv commentA Technical Report
References in corpus (8)
- Bidirectional Attention Flow for Machine Comprehension
- Variational Dropout and the Local Reparameterization Trick
- SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
- Pay Less Attention with Lightweight and Dynamic Convolutions
- Depthwise Separable Convolutions for Neural Machine Translation
- Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
- Fast Reading Comprehension with ConvNets
- Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together