Sent2Matrix: Folding Character Sequences in Serpentine Manifolds for Two-Dimensional Sentence

Abstract

We study text representation methods using deep models. Current methods, such as word-level embedding and character-level embedding schemes, treat texts as either a sequence of atomic words or a sequence of characters. These methods either ignore word morphologies or word boundaries. To overcome these limitations, we propose to convert texts into 2-D representations and develop the Sent2Matrix method. Our method allows for the explicit incorporation of both word morphologies and boundaries. When coupled with a novel serpentine padding method, our Sent2Matrix method leads to an interesting visualization in which 1-D character sequences are folded into 2-D serpentine manifolds. Notably, our method is the first attempt to represent texts in 2-D formats. Experimental results on text classification tasks shown that our method consistently outperforms prior embedding methods.

arXiv comment8 pages

References in corpus (5)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Character-Aware Neural Language Models
• Effective Approaches to Attention-based Neural Machine Translation
• Classifying Relations by Ranking with Convolutional Neural Networks
• Applying Deep Learning to Answer Selection: A Study and An Open Task