Streaming Probabilistic Deep Tensor Factorization
arXiv:2007.07367
Abstract
Despite the success of existing tensor factorization methods, most of them conduct a multilinear decomposition, and rarely exploit powerful modeling frameworks, like deep neural networks, to capture a variety of complicated interactions in data. More important, for highly expressive, deep factorization, we lack an effective approach to handle streaming data, which are ubiquitous in real-world applications. To address these issues, we propose SPIDER, a Streaming ProbabilistIc Deep tEnsoR factorization method. We first use Bayesian neural networks (NNs) to construct a deep tensor factorization model. We assign a spike-and-slab prior over the NN weights to encourage sparsity and prevent overfitting. We then use Taylor expansions and moment matching to approximate the posterior of the NN output and calculate the running model evidence, based on which we develop an efficient streaming posterior inference algorithm in the assumed-density-filtering and expectation propagation framework. Our algorithm provides responsive incremental updates for the posterior of the latent factors and NN weights upon receiving new tensor entries, and meanwhile select and inhibit redundant/useless weights. We show the advantages of our approach in four real-world applications.
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Loopy Belief Propagation for Approximate Inference: An Empirical Study
- Expectation Propagation for approximate Bayesian inference
- Spike and slab variable selection: Frequentist and Bayesian strategies
- Scalable Tensor Factorizations for Incomplete Data
- Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
- Tractable Inference for Complex Stochastic Processes
- Streaming Variational Bayes
- DFacTo: Distributed Factorization of Tensors
- Zero-Truncated Poisson Tensor Factorization for Massive Binary Tensors
- Conditional Expectation Propagation