Supervised Topic Models

Abstract

We introduce supervised latent Dirichlet allocation (sLDA), a statistical model of labelled documents. The model accommodates a variety of response types. We derive an approximate maximum-likelihood procedure for parameter estimation, which relies on variational methods to handle intractable posterior expectations. Prediction problems motivate this research: we use the fitted model to predict response values for new documents. We test sLDA on two real-world problems: movie ratings predicted from reviews, and the political tone of amendments in the U.S. Senate based on the amendment text. We illustrate the benefits of sLDA versus modern regularized regression, as well as versus an unsupervised LDA analysis followed by a separate regression.

References in corpus (1)

• Variational inference for large-scale models of discrete choice

Cited by in corpus (108)

• Recommender Systems
• Task-Driven Dictionary Learning
• Supervised Dictionary Learning
• A Brief Survey of Text Mining: Classification, Clustering and Extraction Techniques
• Topic Models Conditioned on Arbitrary Features with Dirichlet-multinomial Regression
• Proximal Methods for Hierarchical Sparse Coding
• Hierarchical relational models for document networks
• Discovering Discrete Latent Topics with Neural Variational Inference
• A Survey on Data-driven Software Vulnerability Assessment and Prioritization
• Investigating Capsule Networks with Dynamic Routing for Text Classification
• Finding Complex Biological Relationships in Recent PubMed Articles Using Bio-LDA
• Learning Supervised Topic Models for Classification and Regression from Crowds
• Inferring Networks of Substitutable and Complementary Products
• Representation Learning for Natural Language Processing
• Gibbs Max-margin Topic Models with Data Augmentation
• A Deep and Autoregressive Approach for Topic Modeling of Multimodal Data
• Discriminative Topic Mining via Category-Name Guided Text Embedding
• Classification Aware Neural Topic Model and its Application on a New COVID-19 Disinformation Corpus
• TDAM: a Topic-Dependent Attention Model for Sentiment Analysis
• On Completeness-aware Concept-Based Explanations in Deep Neural Networks
• Learning Causal Semantic Representation for Out-of-Distribution Prediction
• Automatic Aggregation by Joint Modeling of Aspects and Values
• The supervised hierarchical Dirichlet process
• Keyword Assisted Topic Models
• A Framework for Optimizing Paper Matching
• Description Based Text Classification with Reinforcement Learning
• User and Item-aware Estimation of Review Helpfulness
• Homogeneity-Based Transmissive Process to Model True and False News in Social Networks
• The Dynamic Embedded Topic Model
• Gene Expression based Survival Prediction for Cancer Patients: A Topic Modeling Approach
• Online Bayesian Passive-Aggressive Learning
• Keyword Assisted Embedded Topic Model
• End-to-end Learning of LDA by Mirror-Descent Back Propagation over a Deep Architecture
• Item Recommendation with Evolving User Preferences and Experience
• Dirichlet Process with Mixed Random Measures: A Nonparametric Topic Model for Labeled Data
• Computer-Assisted Text Analysis for Social Science: Topic Models and Beyond
• Sentence Level Recurrent Topic Model: Letting Topics Speak for Themselves
• Concise comparative summaries (CCS) of large text corpora with a human experiment
• Big Learning with Bayesian Methods
• Recent Advances in Text Analysis
• Connection Discovery using Shared Images by Gaussian Relational Topic Model
• Using Variational Inference and MapReduce to Scale Topic Modeling
• Mining Software Quality from Software Reviews: Research Trends and Open Issues
• Nonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction
• A Spectral Algorithm for Latent Junction Trees
• Enriching Ontologies with Encyclopedic Background Knowledge for Document Indexing
• Transfer Topic Modeling with Ease and Scalability
• Supervised topic models for clinical interpretability
• Topological Feature Based Classification
• Prediction Focused Topic Models via Feature Selection
• Transfer Topic Labeling with Domain-Specific Knowledge Base: An Analysis of UK House of Commons Speeches 1935-2014
• Time-series Insights into the Process of Passing or Failing Online University Courses using Neural-Induced Interpretable Student States
• Prediction-Constrained Topic Models for Antidepressant Recommendation
• Generative Topic Embedding: a Continuous Representation of Documents (Extended Version with Proofs)
• Ordering-sensitive and Semantic-aware Topic Modeling
• Survival-supervised latent Dirichlet allocation models for genomic analysis of time-to-event outcomes
• Hybrid Generative/Discriminative Learning for Automatic Image Annotation
• Supervised Laplacian Eigenmaps with Applications in Clinical Diagnostics for Pediatric Cardiology
• Supervised Blockmodelling
• Auto-clustering Output Layer: Automatic Learning of Latent Annotations in Neural Networks
• Priors for Random Count Matrices Derived from a Family of Negative Binomial Processes
• Prediction-Constrained Training for Semi-Supervised Mixture and Topic Models
• Style-compatible Object Recommendation for Multi-room Indoor Scene Synthesis
• Neural Topic Modeling with Cycle-Consistent Adversarial Training
• Improving unsupervised neural aspect extraction for online discussions using out-of-domain classification
• Multinomial Inverse Regression for Text Analysis
• Generating Synthetic Text Data to Evaluate Causal Inference Methods
• Deep Autoencoding Topic Model with Scalable Hybrid Bayesian Inference
• PreCog: Improving Crowdsourced Data Quality Before Acquisition
• Jointly Modeling Topics and Intents with Global Order Structure
• Clust-LDA: Joint Model for Text Mining and Author Group Inference
• Basic tasks of sentiment analysis
• Community Specific Temporal Topic Discovery from Social Media
• Energy-Based Processes for Exchangeable Data
• Managing sparsity, time, and quality of inference in topic models
• Modeling Word Relatedness in Latent Dirichlet Allocation
• Learning Representations of Social Media Users
• Supervised multi-specialist topic model with applications on large-scale electronic health record data
• A latent topic model for mining heterogenous non-randomly missing electronic health records data
• Augmenting word2vec with latent Dirichlet allocation within a clinical application
• Heron Inference for Bayesian Graphical Models
• Stochastic Discriminative EM
• Discovering topics with neural topic models built from PLSA assumptions
• Geometric Dirichlet Means algorithm for topic inference
• MedLDA: A General Framework of Maximum Margin Supervised Topic Models
• Multi-Aspect Sentiment Analysis with Latent Sentiment-Aspect Attribution
• Interactive Storytelling over Document Collections
• Spectral Learning for Supervised Topic Models
• On Learning Prediction-Focused Mixtures
• Have you tried Neural Topic Models? Comparative Analysis of Neural and Non-Neural Topic Models with Application to COVID-19 Twitter Data
• Deep Poisson Factorization Machines: factor analysis for mapping behaviors in journalist ecosystem
• Dropout Training for SVMs with Data Augmentation
• Learning beyond Predefined Label Space via Bayesian Nonparametric Topic Modelling
• Prediction Focused Topic Models for Electronic Health Records
• Interpretable Low-Rank Document Representations with Label-Dependent Sparsity Patterns
• Multiple Location Profiling for Users and Relationships from Social Network and Content
• Interactive Open-Ended Learning for 3D Object Recognition
• Communication-Free Parallel Supervised Topic Models
• Metadata Enrichment of Multi-Disciplinary Digital Library: A Semantic-based Approach
• ALBU: An approximate Loopy Belief message passing algorithm for LDA to improve performance on small data sets
• Triple Scoring Using Paragraph Vector - The Gailan Triple Scorer at WSDM Cup 2017
• Weakly Supervised Domain Detection
• Topics to Avoid: Demoting Latent Confounds in Text Classification
• Neural Topic Models with Survival Supervision: Jointly Predicting Time-to-Event Outcomes and Learning How Clinical Features Relate
• Modeling Online Reviews with Multi-grain Topic Models
• Few-shot Learning for Topic Modeling
• Inter-Battery Topic Representation Learning
• Conceptualization Topic Modeling