Efficient Purely Convolutional Text Encoding

Abstract

In this work, we focus on a lightweight convolutional architecture that creates fixed-size vector embeddings of sentences. Such representations are useful for building NLP systems, including conversational agents. Our work derives from a recently proposed recursive convolutional architecture for auto-encoding text paragraphs at byte level. We propose alternations that significantly reduce training time, the number of parameters, and improve auto-encoding accuracy. Finally, we evaluate the representations created by our model on tasks from SentEval benchmark suite, and show that it can serve as a better, yet fairly low-resource alternative to popular bag-of-words embeddings.

arXiv commentAs accepted to: LaCATODA Workshop, ICML 2018

References in corpus (10)

• Efficient Estimation of Word Representations in Vector Space
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Axiomatic Attribution for Deep Networks
• Convolutional Sequence to Sequence Learning
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• A large annotated corpus for learning natural language inference
• Enriching Word Vectors with Subword Information
• Learning Semantic Textual Similarity from Conversations
• RubyStar: A Non-Task-Oriented Mixture Model Dialog System