Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity
arXiv:0911.0054
Abstract
The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions, such as when there is some sparsity pattern of the optimal parameter. This work characterizes a certain strong convexity property of general exponential families, which allow their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L_1 regularization.
Errata added. Incorrect claim about cumulants of the Bernoulli distribution fixed
Cited by in corpus (16)
- A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
- A modern maximum-likelihood theory for high-dimensional logistic regression
- Classification vs regression in overparameterized regimes: Does the loss function matter?
- Learning Model-Based Sparsity via Projected Gradient Descent
- Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
- Exponential Family Matrix Completion under Structural Constraints
- The Impact of Regularization on High-dimensional Logistic Regression
- Learning Latent Variable Gaussian Graphical Models
- Minimax Optimal Sparse Signal Recovery with Poisson Statistics
- Sparse Signal Recovery under Poisson Statistics
- Exponential Series Approaches for Nonparametric Graphical Models
- Reconstruction Error Bounds for Compressed Sensing under Poisson or Poisson-Gaussian Noise Using Variance Stabilization Transforms
- Limit theorems for dependent combinatorial data, with applications in statistical inference
- A Computationally Efficient Method for Learning Exponential Family Distributions
- Reconstruction Error Bounds for Compressed Sensing under Poisson Noise using the Square Root of the Jensen-Shannon Divergence
- High Dimensional Logistic Regression Under Network Dependence