Tensor Decompositions in Deep Learning

Abstract

The paper surveys the topic of tensor decompositions in modern machine learning applications. It focuses on three active research topics of significant relevance for the community. After a brief review of consolidated works on multi-way data analysis, we consider the use of tensor decompositions in compressing the parameter space of deep learning models. Lastly, we discuss how tensor methods can be leveraged to yield richer adaptive representations of complex data, including structured information. The paper concludes with a discussion on interesting open research challenges.

References in corpus (6)

• Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
• A Gentle Introduction to Deep Learning for Graphs
• Equivalence of restricted Boltzmann machines and tensor network states
• TensorLy: Tensor Learning in Python
• A Fair Comparison of Graph Neural Networks for Graph Classification
• Deep Multi-task Representation Learning: A Tensor Factorisation Approach