Lower and Upper Bounds on the VC-Dimension of Tensor Network Models
arXiv:2106.11827
Abstract
Tensor network methods have been a key ingredient of advances in condensed matter physics and have recently sparked interest in the machine learning community for their ability to compactly represent very high-dimensional objects. Tensor network methods can for example be used to efficiently learn linear models in exponentially large feature spaces [Stoudenmire and Schwab, 2016]. In this work, we derive upper and lower bounds on the VC dimension and pseudo-dimension of a large class of tensor network models for classification, regression and completion. Our upper bounds hold for linear models parameterized by arbitrary tensor network structures, and we derive lower bounds for common tensor decomposition models~(CP, Tensor Train, Tensor Ring and Tucker) showing the tightness of our general upper bound. These results are used to derive a generalization bound which can be applied to classification with low rank matrices as well as linear classifiers based on any of the commonly used tensor decomposition models. As a corollary of our results, we obtain a bound on the VC dimension of the matrix product state classifier introduced in [Stoudenmire and Schwab, 2016] as a function of the so-called bond dimension~(i.e. tensor train rank), which answers an open problem listed by Cirac, Garre-Rubio and Pérez-García in [Cirac et al., 2019].
References in corpus (18)
- Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
- Matrix Product States, Projected Entangled Pair States, and variational renormalization group methods for quantum spin systems
- Scalable Tensor Factorizations for Incomplete Data
- Tensor Ring Decomposition
- Unsupervised Generative Modeling Using Matrix Product States
- Tensor Networks in a Nutshell
- Convex Tensor Decomposition via Structured Schatten Norm Regularization
- Supervised Learning with Projected Entangled Pair States
- Optimal Estimation of Low Rank Density Matrices
- Tensor Completion by Alternating Minimization under the Tensor Train (TT) Model
- Wide Compression: Tensor Ring Nets
- Efficient tensor completion: Low-rank tensor train
- Asymptotic equivalence of quantum state tomography and noisy matrix completion
- Mathematical open problems in Projected Entangled Pair States
- Learning with tree tensor networks: complexity estimates and model selection
- Tensor Networks for Medical Image Classification
- On Low-rank Trace Regression under General Sampling Distribution
- Partial Trace Regression and Low-Rank Kraus Decomposition