Nonnegative Matrix Factorization and I-Divergence Alternating Minimization

Abstract

In this paper we consider the Nonnegative Matrix Factorization (NMF) problem: given an (elementwise) nonnegative matrix $V\in {\mathbb{R}}_{+}^{m\times n}$ find, for assigned $k$, nonnegative matrices $W\in {\mathbb{R}}_{+}^{m\times k}$ and $H\in {\mathbb{R}}_{+}^{k\times n}$ such that $V=WH$. Exact, non trivial, nonnegative factorizations do not always exist, hence it is interesting to pose the approximate NMF problem. The criterion which is commonly employed is I-divergence between nonnegative matrices. The problem becomes that of finding, for assigned $k$, the factorization $WH$ closest to $V$ in I-divergence. An iterative algorithm, EM like, for the construction of the best pair $(W,H)$ has been proposed in the literature. In this paper we interpret the algorithm as an alternating minimization procedure à la Csiszár-Tusnády and investigate some of its stability properties. NMF is widespreading as a data analysis method in applications for which the positivity constraint is relevant. There are other data analysis methods which impose some form of nonnegativity: we discuss here the connections between NMF and Archetypal Analysis.

References in corpus (1)

• Approximate Nonnegative Matrix Factorization via Alternating Minimization

Cited by in corpus (13)

• On Tensors, Sparsity, and Nonnegative Factorizations
• Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
• Algorithms for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence
• Approximation of stationary processes by Hidden Markov Models
• Upper Bound of Bayesian Generalization Error in Non-negative Matrix Factorization
• The inverse problem of positive autoconvolution
• Approximation of Nonnegative Systems by Finite Impulse Response Convolutions
• Bayesian Allocation Model: Inference by Sequential Monte Carlo for Nonnegative Tensor Factorizations and Topic Models using Polya Urns
• Majorization-minimization Bregman proximal gradient algorithms for NMF with the Kullback--Leibler divergence
• Approximate factor analysis model building via alternating I-divergence minimization
• Approximation of nonnegative systems by moving averages of fixed order
• Two-step Nonnegative Matrix Factorization Algorithm for the Approximate Realization of Hidden Markov Models
• Synchronous deautoconvolution algorithm for discrete-time positive signals via I-divergence approximation