Stochastic Learning of Nonstationary Kernels for Natural Language Modeling
arXiv:1801.03911
Abstract
Natural language processing often involves computations with semantic or syntactic graphs to facilitate sophisticated reasoning based on structural relationships. While convolution kernels provide a powerful tool for comparing graph structure based on node (word) level relationships, they are difficult to customize and can be computationally expensive. We propose a generalization of convolution kernels, with a nonstationary model, for better expressibility of natural languages in supervised settings. For a scalable learning of the parameters introduced with our model, we propose a novel algorithm that leverages stochastic sampling on k-nearest neighbor graphs, along with approximations based on locality-sensitive hashing. We demonstrate the advantages of our approach on a challenging real-world (structured inference) problem of automatically extracting biological models from the text of scientific papers.
References in corpus (12)
- Sequence to Sequence Learning with Neural Networks
- Submodular meets Spectral: Greedy Algorithms for Subset Selection, Sparse Approximation and Dictionary Selection
- Learning Longer Memory in Recurrent Neural Networks
- End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures
- When Are Tree Structures Necessary for Deep Learning of Representations?
- Classifying Relations by Ranking with Convolutional Neural Networks
- Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path
- Heteroscedastic Treed Bayesian Optimisation
- Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling
- Extracting Biomolecular Interactions Using Semantic Parsing of Biomedical Text
- Task-Oriented Learning of Word Embeddings for Semantic Relation Classification
- Learning Structural Kernels for Natural Language Processing