Convolutional Neural Network Architectures for Matching Natural Language Sentences

Abstract

Semantic matching is of central importance to many natural language tasks \cite{bordes2014semantic,RetrievalQA}. A successful matching algorithm needs to adequately model the internal structures of language objects and the interaction between them. As a step toward this goal, we propose convolutional neural network models for matching two sentences, by adapting the convolutional strategy in vision and speech. The proposed models not only nicely represent the hierarchical structures of sentences with their layer-by-layer composition and pooling, but also capture the rich matching patterns at different levels. Our models are rather generic, requiring no prior knowledge on language, and can hence be applied to matching tasks of different nature and in different languages. The empirical study on a variety of matching tasks demonstrates the efficacy of the proposed model on a variety of matching tasks and its superiority to competitor models.

References in corpus (3)

• Efficient Estimation of Word Representations in Vector Space
• Improving neural networks by preventing co-adaptation of feature detectors
• Natural Language Processing (almost) from Scratch

Cited by in corpus (220)

• Hierarchical Question-Image Co-Attention for Visual Question Answering
• A Deep Relevance Matching Model for Ad-hoc Retrieval
• Reasoning about Entailment with Neural Attention
• A Survey on Dialogue Systems: Recent Advances and New Frontiers
• End-to-End Neural Ad-hoc Ranking with Kernel Pooling
• LSTM-based Deep Learning Models for Non-factoid Answer Selection
• Attentive Pooling Networks
• Representation learning for very short texts using weighted word embedding aggregation
• Natural Language Processing Advancements By Deep Learning: A Survey
• A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations
• Text Matching as Image Recognition
• Question Answering and Question Generation as Dual Tasks
• Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems
• Match-SRNN: Modeling the Recursive Matching Structure with Spatial RNN
• Adversarial Feature Matching for Text Generation
• A Deep Network Model for Paraphrase Detection in Short Text Messages
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Towards Universal Paraphrastic Sentence Embeddings
• Classifying Relations by Ranking with Convolutional Neural Networks
• Modeling Diverse Relevance Patterns in Ad-hoc Retrieval
• Building Damage Annotation on Post-Hurricane Satellite Imagery Based on Convolutional Neural Networks
• Learning Semantic Similarity for Very Short Texts
• Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching
• Extraction of Salient Sentences from Labelled Documents
• Neural Models for Information Retrieval
• How Transferable are Neural Networks in NLP Applications?
• A Study of MatchPyramid Models on Ad-hoc Retrieval
• Applying Deep Learning to Answer Selection: A Study and An Open Task
• Adversarial Neural Machine Translation
• Graph Neural Networks for Natural Language Processing: A Survey
• Deconvolutional Paragraph Representation Learning
• Discriminative Neural Sentence Modeling by Tree-Based Convolution
• A Decomposable Attention Model for Natural Language Inference
• Towards Neural Network-based Reasoning
• RepBERT: Contextualized Text Embeddings for First-Stage Retrieval
• Natural Language Understanding with the Quora Question Pairs Dataset
• Conversational Contextual Cues: The Case of Personalization and History for Response Ranking
• ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
• Strong error analysis for stochastic gradient descent optimization algorithms
• Encoding Source Language with Convolutional Neural Network for Machine Translation
• Quantum Language Model with Entanglement Embedding for Question Answering
• Self-Adaptive Hierarchical Sentence Model
• Learning text representation using recurrent convolutional neural network with highway layers
• aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model
• Natural Language Inference by Tree-Based Convolution and Heuristic Matching
• Modeling Relational Information in Question-Answer Pairs with Convolutional Neural Networks
• Neural Information Retrieval: A Literature Review
• Challenges in Building Intelligent Open-domain Dialog Systems
• Using Prior Knowledge to Guide BERT's Attention in Semantic Textual Matching Tasks
• Neural ranking models for document retrieval
• Multiresolution Graph Attention Networks for Relevance Matching
• A Multi-cascaded Model with Data Augmentation for Enhanced Paraphrase Detection in Short Texts
• MatchZoo: A Toolkit for Deep Text Matching
• RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm
• StruBERT: Structure-aware BERT for Table Search and Matching
• Deep Multi-Task Learning with Shared Memory
• Neural Generative Question Answering
• Learning a Product Relevance Model from Click-Through Data in E-Commerce
• Learning to Extract Coherent Summary via Deep Reinforcement Learning
• CNM: An Interpretable Complex-valued Network for Matching
• Syntax-based Deep Matching of Short Texts
• Diagnosing BERT with Retrieval Heuristics
• SemEval-2021 Task 11: NLPContributionGraph -- Structuring Scholarly NLP Contributions for a Research Knowledge Graph
• Learning to Answer Questions From Image Using Convolutional Neural Network
• Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems
• Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
• Asynchronous Federated Learning with Differential Privacy for Edge Intelligence
• How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence
• Answer Sequence Learning with Neural Networks for Answer Selection in Community Question Answering
• Multi-Cast Attention Networks for Retrieval-based Question Answering and Response Prediction
• LCSTS: A Large Scale Chinese Short Text Summarization Dataset
• DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modeling
• A Hybrid Retrieval-Generation Neural Conversation Model
• Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
• Paraphrase Generation with Deep Reinforcement Learning
• Machine Learning Approach for Detection of nonTor Traffic
• Match-Ignition: Plugging PageRank into Transformer for Long-form Text Matching
• Local Translation Prediction with Global Sentence Representation
• Multi-Label Learning from Medical Plain Text with Convolutional Residual Models
• Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory
• Deep Relevance Ranking Using Enhanced Document-Query Interactions
• Deep neural network approximations for Monte Carlo algorithms
• Learning Generic Sentence Representations Using Convolutional Neural Networks
• SimGNN: A Neural Network Approach to Fast Graph Similarity Computation
• FlipDial: A Generative Model for Two-Way Visual Dialogue
• The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
• SetRank: Learning a Permutation-Invariant Ranking Model for Information Retrieval
• ATOM: Commit Message Generation Based on Abstract Syntax Tree and Hybrid Ranking
• Learning To Retrieve: How to Train a Dense Retrieval Model Effectively and Efficiently
• Automatic Judgment Prediction via Legal Reading Comprehension
• A Deep Look into Neural Ranking Models for Information Retrieval
• Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models
• Semantic Product Search
• Learning Contextualized Document Representations for Healthcare Answer Retrieval
• Fine-grained Event Categorization with Heterogeneous Graph Convolutional Networks
• Cross-domain Semantic Parsing via Paraphrasing
• Learning to Match Using Local and Distributed Representations of Text for Web Search
• Modelling Interaction of Sentence Pair with coupled-LSTMs
• Text-to-Viz: Automatic Generation of Infographics from Proportion-Related Natural Language Statements
• Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
• A Deep Investigation of Deep IR Models
• $gen$CNN: A Convolutional Architecture for Word Sequence Prediction
• A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data using Unstructured Spatial Discretizations
• Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce
• Automated Steel Bar Counting and Center Localization with Convolutional Neural Networks
• DAL: Dual Adversarial Learning for Dialogue Generation
• Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues
• Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots
• Cross Temporal Recurrent Networks for Ranking Question Answer Pairs
• Context-Dependent Translation Selection Using Convolutional Neural Network
• A Fast Deep Learning Model for Textual Relevance in Biomedical Information Retrieval
• Matching Algorithms: Fundamentals, Applications and Challenges
• A Comparative Study on Regularization Strategies for Embedding-based Neural Networks
• Neural IR Meets Graph Embedding: A Ranking Model for Product Search
• A Context-Aware User-Item Representation Learning for Item Recommendation
• Deep Binaries: Encoding Semantic-Rich Cues for Efficient Textual-Visual Cross Retrieval
• Charge-Based Prison Term Prediction with Deep Gating Network
• Toward a Deep Neural Approach for Knowledge-Based IR
• Dropout Prediction over Weeks in MOOCs via Interpretable Multi-Layer Representation Learning
• Learning a Matching Model with Co-teaching for Multi-turn Response Selection in Retrieval-based Dialogue Systems
• Co-PACRR: A Context-Aware Neural IR Model for Ad-hoc Retrieval
• Promotion of Answer Value Measurement with Domain Effects in Community Question Answering Systems
• Building Data-driven Models with Microstructural Images: Generalization and Interpretability
• Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots
• A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots
• Matching Article Pairs with Graphical Decomposition and Convolutions
• Lattice CNNs for Matching Based Chinese Question Answering
• Learning Semantically Coherent and Reusable Kernels in Convolution Neural Nets for Sentence Classification
• CORE: Automating Review Recommendation for Code Changes
• Neural Network Based Next-Song Recommendation
• The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection
• M${}^2$S-Net: Multi-Modal Similarity Metric Learning based Deep Convolutional Network for Answer Selection
• A Document-grounded Matching Network for Response Selection in Retrieval-based Chatbots
• CorDEL: A Contrastive Deep Learning Approach for Entity Linkage
• Modeling Document Interactions for Learning to Rank with Regularized Self-Attention
• A Comparison of Supervised Learning to Match Methods for Product Search
• Fine-Grained Sentence Functions for Short-Text Conversation
• LATTE: Latent Type Modeling for Biomedical Entity Linking
• Learning Context-Sensitive Convolutional Filters for Text Processing
• An Empirical Study of Language CNN for Image Captioning
• GlyphCRM: Bidirectional Encoder Representation for Chinese Character with its Glyph
• BanditRank: Learning to Rank Using Contextual Bandits
• Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
• Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots
• Combining Spiking Neural Network and Artificial Neural Network for Enhanced Image Classification
• Heterogeneous Network Embedding for Deep Semantic Relevance Match in E-commerce Search
• Target Guided Emotion Aware Chat Machine
• IART: Intent-aware Response Ranking with Transformers in Information-seeking Conversation Systems
• Graph-augmented Learning to Rank for Querying Large-scale Knowledge Graph
• Learning Fine-grained Fact-Article Correspondence in Legal Cases
• Activation functions are not needed: the ratio net
• How to Build User Simulators to Train RL-based Dialog Systems
• Word Embedding based Edit Distance
• Context Attentive Document Ranking and Query Suggestion
• Why and How to Pay Different Attention to Phrase Alignments of Different Intensities
• Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce
• Comparing Attention-based Convolutional and Recurrent Neural Networks: Success and Limitations in Machine Reading Comprehension
• Identifying High-Quality Chinese News Comments Based on Multi-Target Text Matching Model
• Content Selection Network for Document-grounded Retrieval-based Chatbots
• Chinese Medical Question Answer Matching Based on Interactive Sentence Representation Learning
• FAQ-based Question Answering via Knowledge Anchors
• A Multi-Size Neural Network with Attention Mechanism for Answer Selection
• Toward the Understanding of Deep Text Matching Models for Information Retrieval
• Luandri: a Clean Lua Interface to the Indri Search Engine
• Learning Word Embeddings from Intrinsic and Extrinsic Views
• Self-supervised Answer Retrieval on Clinical Notes
• Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
• Visual Question Answering as a Multi-Task Problem
• Biomedical Evidence Generation Engine
• TraceWalk: Semantic-based Process Graph Embedding for Consistency Checking
• COMPOSE: Cross-Modal Pseudo-Siamese Network for Patient Trial Matching
• Impact of Training Dataset Size on Neural Answer Selection Models
• A Social Search Model for Large Scale Social Networks
• Retrieve Synonymous keywords for Frequent Queries in Sponsored Search in a Data Augmentation Way
• Geometric Operator Convolutional Neural Network
• Deploying Deep Ranking Models for Search Verticals
• MRNN: A Multi-Resolution Neural Network with Duplex Attention for Document Retrieval in the Context of Question Answering
• Optimize What You Evaluate With: A Simple Yet Effective Framework For Direct Optimization Of IR Metrics
• WIKIR: A Python toolkit for building a large-scale Wikipedia-based English Information Retrieval Dataset
• Neural Article Pair Modeling for Wikipedia Sub-article Matching
• Neural Relation Prediction for Simple Question Answering over Knowledge Graph
• GLOW : Global Weighted Self-Attention Network for Web Search
• BiTe-GCN: A New GCN Architecture via BidirectionalConvolution of Topology and Features on Text-Rich Networks
• Spherical Convolutional Neural Networks: Stability to Perturbations in SO(3)
• Inducing Alignment Structure with Gated Graph Attention Networks for Sentence Matching
• Wasserstein Distance Regularized Sequence Representation for Text Matching in Asymmetrical Domains
• An In-depth Analysis of Passage-Level Label Transfer for Contextual Document Ranking
• Click-Based Student Performance Prediction: A Clustering Guided Meta-Learning Approach
• Identical and Fraternal Twins: Fine-Grained Semantic Contrastive Learning of Sentence Representations
• Is it a click bait? Let's predict using Machine Learning
• A review for Tone-mapping Operators on Wide Dynamic Range Image
• Can AI Generate Love Advice?: Toward Neural Answer Generation for Non-Factoid Questions
• Character-level Deep Conflation for Business Data Analytics
• An Intelligent CNN-VAE Text Representation Technology Based on Text Semantics for Comprehensive Big Data
• Robustness to Modification with Shared Words in Paraphrase Identification
• CONNA: Addressing Name Disambiguation on The Fly
• Learning to Embed Sentences Using Attentive Recursive Trees
• Automatic Question-Answering Using A Deep Similarity Neural Network
• Towards Deep and Representation Learning for Talent Search at LinkedIn
• Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations
• Freudian and Newtonian Recurrent Cell for Sequential Recommendation
• Mapping Natural Language Commands to Web Elements
• A scalable framework for learning from implicit user feedback to improve natural language understanding in large-scale conversational AI systems
• Using the Hammer Only on Nails: A Hybrid Method for Evidence Retrieval for Question Answering
• Streaming Social Event Detection and Evolution Discovery in Heterogeneous Information Networks
• Place Deduplication with Embeddings
• PT-Ranking: A Benchmarking Platform for Neural Learning-to-Rank
• Modularized Transfomer-based Ranking Framework
• Improving Matching Models with Hierarchical Contextualized Representations for Multi-turn Response Selection
• MIRA: Leveraging Multi-Intention Co-click Information in Web-scale Document Retrieval using Deep Neural Networks
• Learning to Represent Bilingual Dictionaries
• Unsupervised Context Rewriting for Open Domain Conversation
• Attentive Long Short-Term Preference Modeling for Personalized Product Search
• CASE: Context-Aware Semantic Expansion
• Hierarchical RNN with Static Sentence-Level Attention for Text-Based Speaker Change Detection
• Differentially Private n-gram Extraction
• Graph-based Multilingual Product Retrieval in E-commerce Search
• Embracing Structure in Data for Billion-Scale Semantic Product Search
• BERT-DRE: BERT with Deep Recursive Encoder for Natural Language Sentence Matching
• Learning Fast Matching Models from Weak Annotations