Rademacher Random Projections with Tensor Networks
arXiv:2110.13970
Abstract
Random projection (RP) have recently emerged as popular techniques in the machine learning community for their ability in reducing the dimension of very high-dimensional tensors. Following the work in [30], we consider a tensorized random projection relying on Tensor Train (TT) decomposition where each element of the core tensors is drawn from a Rademacher distribution. Our theoretical results reveal that the Gaussian low-rank tensor represented in compressed form in TT format in [30] can be replaced by a TT tensor with core elements drawn from a Rademacher distribution with the same embedding size. Experiments on synthetic data demonstrate that tensorized Rademacher RP can outperform the tensorized Gaussian RP studied in [30]. In addition, we show both theoretically and experimentally, that the tensorized RP in the Matrix Product Operator (MPO) format is not a Johnson-Lindenstrauss transform (JLT) and therefore not a well-suited random projection map
References in corpus (7)
- Tensor Decomposition for Signal Processing and Machine Learning
- Tensorizing Neural Networks
- Tensor Regression with Applications in Neuroimaging Data Analysis
- Unsupervised Generative Modeling Using Matrix Product States
- Supervised Learning with Quantum-Inspired Tensor Networks
- Guarantees for the Kronecker Fast Johnson-Lindenstrauss Transform Using a Coherence and Sampling Argument
- Tensor Random Projection for Low Memory Dimension Reduction