Identifiability of parameters in latent structure models with many observed variables

Abstract

While hidden class models of various types arise in many statistical applications, it is often difficult to establish the identifiability of their parameters. Focusing on models in which there is some structure of independence of some of the observed variables conditioned on hidden ones, we demonstrate a general approach for establishing identifiability utilizing algebraic arguments. A theorem of J. Kruskal for a simple latent-class model with finite state space lies at the core of our results, though we apply it to a diverse set of models. These include mixtures of both finite and nonparametric product distributions, hidden Markov models and random graph mixture models, and lead to a number of new results and improvements to old ones. In the parametric setting, this approach indicates that for such models, the classical definition of identifiability is typically too strong. Instead generic identifiability holds, which implies that the set of nonidentifiable parameters has measure zero, so that parameter inference is still meaningful. In particular, this sheds light on the properties of finite mixtures of Bernoulli products, which have been used for decades despite being known to have nonidentifiable parameters. In the nonparametric setting, we again obtain identifiability only when certain restrictions are placed on the distributions that are mixed, but we explicitly describe the conditions.

Published in at http://dx.doi.org/10.1214/09-AOS689 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• Algebraic Techniques for Gaussian Models

Cited by in corpus (92)

• Overlapping stochastic block models with application to the French political blogosphere
• Genome scans for detecting footprints of local adaptation using a Bayesian factor model
• An algorithm for generic and low-rank specific identifiability of complex tensors
• Block Models and Personalized PageRank
• Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data
• The Hitchhiker guide to: Secant Varieties and Tensor Decomposition
• On the Identifiability of Diagnostic Classification Models
• Model-based clustering for conditionally correlated categorical data
• The condition number of join decompositions
• Fixed points of the EM algorithm and nonnegative rank boundaries
• Margins of discrete Bayesian networks
• Estimating multivariate latent-structure models
• Bounded-rank tensors are defined in bounded degree
• A Riemannian Trust Region Method for the Canonical Tensor Rank Approximation Problem
• Convergence of the groups posterior distribution in latent or stochastic block models
• A condition number for the tensor rank decomposition
• Tensor ranks on tangent developable of Segre varieties
• On Multi-Cause Causal Inference with Unobserved Confounding: Counterexamples, Impossibility, and Alternatives
• A two-step estimator for multilevel latent class analysis with covariates
• About the posterior distribution in hidden Markov models with unknown number of states
• Estimating Discrete Markov Models From Various Incomplete Data Schemes
• Identifiability of restricted latent class models with binary responses
• Unsupervised Learning of Noisy-Or Bayesian Networks
• Identifiability and Unmixing of Latent Parse Trees
• Identifiability beyond Kruskal's bound for symmetric tensors of degree 4
• Contrastive learning, multi-view redundancy, and linear models
• Partial Identifiability of Restricted Latent Class Models
• Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series
• Sufficient and Necessary Conditions for the Identifiability of the $Q$-matrix
• Identification of discrete concentration graph models with one hidden binary variable
• When are Overcomplete Topic Models Identifiable? Uniqueness of Tensor Tucker Decompositions with Structured Sparsity
• Learning Attribute Patterns in High-Dimensional Structured Latent Attribute Models
• Finite state space non parametric Hidden Markov Models are in general identifiable
• Tree cumulants and the geometry of binary tree models
• On the identifiability of ternary forms
• Minimax adaptive estimation of nonparametric hidden Markov models
• Low-rank Characteristic Tensor Density Estimation Part I: Foundations
• A Separation Theorem for Chain Event Graphs
• Learning Mixtures of Plackett-Luce Models
• Towards Clarifying the Theory of the Deconfounder
• Grassmann secants, identifiability, and linear systems of tensors
• A Tensor-EM Method for Large-Scale Latent Class Analysis with Binary Responses
• Mixture of hidden Markov models for accelerometer data
• Multiresolution Tensor Learning for Efficient and Interpretable Spatial Analysis
• Identifiability of Hierarchical Latent Attribute Models
• Clustering Via Finite Nonparametric ICA Mixture Models
• Dimension-Grouped Mixed Membership Models for Multivariate Categorical Data
• Non parametric finite translation mixtures with dependent regime
• Identifiability of Latent Class Models with Covariates
• Identifiability for a class of symmetric tensors
• Identifying Finite Mixtures of Nonparametric Product Distributions and Causal Inference of Confounders
• Latent Theme Dictionary Model for Finding Co-occurrent Patterns in Process Data
• Fundamental limits for learning hidden Markov model parameters
• Intact-VAE: Estimating Treatment Effects under Unobserved Confounding
• Independent Innovation Analysis for Nonlinear Vector Autoregressive Process
• Finite mixture model of conditional dependencies modes to cluster categorical data
• Discussion of "The Blessings of Multiple Causes" by Wang and Blei
• The average condition number of most tensor rank decomposition problems is infinite
• New consistent and asymptotically normal estimators for random graph mixture models
• Multi-Resolution Weak Supervision for Sequential Data
• Latent-state models for precision medicine
• A Solution for Large Scale Nonlinear Regression with High Rank and Degree at Constant Memory Complexity via Latent Tensor Reconstruction
• Tensor Completion Made Practical
• A semiparametric extension of the stochastic block model for longitudinal networks
• On the average condition number of tensor rank decompositions
• Model-based clustering in simple hypergraphs through a stochastic blockmodel
• Memory capacity of two layer neural networks with smooth activations
• Minimality and uniqueness for decompositions of specific ternary forms
• Sufficient and Necessary Conditions for the Identifiability of DINA Models with Polytomous Responses
• Learning latent causal graphs via mixture oracles
• Translation-invariant functional clustering on COVID-19 deaths adjusted on population risk factors
• Community Detection in Weighted Multilayer Networks with Ambient Noise
• A Robbins Monro algorithm for nonparametric estimation of NAR process with Markov-Switching: consistency
• Learning from Multiple Noisy Partial Labelers
• Robust Estimation in Finite Mixture Models
• Uniform Consistency in Nonparametric Mixture Models
• Composition-Aware Spectroscopic Tomography
• Riemannian Newton optimization methods for the symmetric tensor approximation problem
• Source Identification for Mixtures of Product Distributions
• Stealthy hacking and secrecy of controlled state estimation systems with random dropouts
• On a geometric method for the identifiability of forms
• Efficient Tensor Decomposition
• Learning Parametric-Output HMMs with Two Aliased States
• Clustering Data with Nonignorable Missingness using Semi-Parametric Mixture Models
• Learning from Positive and Unlabeled Data by Identifying the Annotation Process
• A Comprehensive Framework for the Evaluation of Individual Treatment Rules From Observational Data
• Identifiability of parametric random matrix models
• Information Theoretic Bounds on Optimal Worst-case Error in Binary Mixture Identification
• Dependent Latent Class Models
• Adaptive Multi-Source Causal Inference
• Quadratic number of nodes is sufficient to learn a dataset via gradient descent
• Observational nonidentifiability, generalized likelihood and free energy