An attentive neural architecture for joint segmentation and parsing and its application to real estate ads
arXiv:1709.09590 · doi:10.1016/j.eswa.2018.02.031
Abstract
In processing human produced text using natural language processing (NLP) techniques, two fundamental subtasks that arise are (i) segmentation of the plain text into meaningful subunits (e.g., entities), and (ii) dependency parsing, to establish relations between subunits. In this paper, we develop a relatively simple and effective neural joint model that performs both segmentation and dependency parsing together, instead of one after the other as in most state-of-the-art works. We will focus in particular on the real estate ad setting, aiming to convert an ad to a structured description, which we name property tree, comprising the tasks of (1) identifying important entities of a property (e.g., rooms) from classifieds and (2) structuring them into a tree format. In this work, we propose a new joint model that is able to tackle the two tasks simultaneously and construct the property tree by (i) avoiding the error propagation that would arise from the subtasks one after the other in a pipelined fashion, and (ii) exploiting the interactions between the subtasks. For this purpose, we perform an extensive comparative study of the pipeline methods and the new proposed joint model, reporting an improvement of over three percentage points in the overall edge F1 score of the property tree. Also, we propose attention methods, to encourage our model to focus on salient tokens during the construction of the property tree. Thus we experimentally demonstrate the usefulness of attentive neural architectures for the proposed joint model, showcasing a further improvement of two percentage points in edge F1 score for our application.
arXiv commentPreprint - Accepted for publication in Expert Systems with Applications
References in corpus (11)
- Sequence to Sequence Learning with Neural Networks
- TensorFlow: A system for large-scale machine learning
- Natural Language Processing (almost) from Scratch
- Neural Message Passing for Quantum Chemistry
- On the difficulty of training Recurrent Neural Networks
- Effective Approaches to Attention-based Neural Machine Translation
- Stochastic Configuration Networks: Fundamentals and Algorithms
- Transition-Based Dependency Parsing with Stack Long Short-Term Memory
- End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures
- Classifying Relations by Ranking with Convolutional Neural Networks
- An autoregressive approach to house price modeling
Cited by in corpus (4)
- Joint entity recognition and relation extraction as a multi-head selection problem
- Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
- Sub-event detection from Twitter streams as a sequence labeling problem
- Joint Extraction of Entity and Relation with Information Redundancy Elimination