Learning Efficient Tensor Representations with Ring Structure Networks
arXiv:1705.08286
Abstract
Tensor train (TT) decomposition is a powerful representation for high-order tensors, which has been successfully applied to various machine learning tasks in recent years. However, since the tensor product is not commutative, permutation of data dimensions makes solutions and TT-ranks of TT decomposition inconsistent. To alleviate this problem, we propose a permutation symmetric network structure by employing circular multilinear products over a sequence of low-order core tensors. This network structure can be graphically interpreted as a cyclic interconnection of tensors, and thus we call it tensor ring (TR) representation. We develop several efficient algorithms to learn TR representation with adaptive TR-ranks by employing low-rank approximations. Furthermore, mathematical properties are investigated, which enables us to perform basic operations in a computationally efficiently way by using TR representations. Experimental results on synthetic signals and real-world datasets demonstrate that the proposed TR network is more expressive and consistently informative than existing TT networks.
arXiv admin note: substantial text overlap with arXiv:1606.05535
Cited by in corpus (9)
- Tensor Methods in Computer Vision and Deep Learning
- Provable Tensor Ring Completion
- Randomized Tensor Ring Decomposition and Its Application to Large-scale Data Reconstruction
- Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
- Learning Compact Recurrent Neural Networks with Block-Term Tensor Decomposition
- Higher-dimension Tensor Completion via Low-rank Tensor Ring Decomposition
- High-dimension Tensor Completion via Gradient-based Optimization Under Tensor-train Format
- Low-rank Tensor Grid for Image Completion
- A Unified Framework for Coupled Tensor Completion