Modeling Psychotherapy Dialogues with Kernelized Hashcode Representations: A Nonparametric Information-Theoretic Approach
arXiv:1804.10188
Abstract
We propose a novel dialogue modeling framework, the first-ever nonparametric kernel functions based approach for dialogue modeling, which learns kernelized hashcodes as compressed text representations; unlike traditional deep learning models, it handles well relatively small datasets, while also scaling to large ones. We also derive a novel lower bound on mutual information, used as a model-selection criterion favoring representations with better alignment between the utterances of participants in a collaborative dialogue setting, as well as higher predictability of the generated responses. As demonstrated on three real-life datasets, including prominently psychotherapy sessions, the proposed approach significantly outperforms several state-of-art neural network based dialogue systems, both in terms of computational efficiency, reducing training time from days or weeks to hours, and the response quality, achieving an order of magnitude improvement over competitors in frequency of being chosen as the best model by human evaluators.
arXiv commentResponse generative based model added, along with human evaluation
References in corpus (11)
- Estimating Mutual Information
- MINE: Mutual Information Neural Estimation
- Deep Variational Information Bottleneck
- Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization
- Hierarchical Recurrent Attention Network for Response Generation
- Does Dirichlet Prior Smoothing Solve the Shannon Entropy Estimation Problem?
- Maximum Likelihood Estimation of Functionals of Discrete Distributions
- Generative Deep Neural Networks for Dialogue: A Short Review
- Relevant sparse codes with variational information bottleneck
- Variational Information Maximization for Feature Selection
- Nearly-Unsupervised Hashcode Representations for Relation Extraction