Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions -- Part II

Abstract

We study the approximation by tensor networks (TNs) of functions from classical smoothness classes. The considered approximation tool combines a tensorization of functions in $L^p([0,1))$, which allows to identify a univariate function with a multivariate function (or tensor), and the use of tree tensor networks (the tensor train format) for exploiting low-rank structures of multivariate functions. The resulting tool can be interpreted as a feed-forward neural network, with first layers implementing the tensorization, interpreted as a particular featuring step, followed by a sum-product network with sparse architecture. In part I of this work, we presented several approximation classes associated with different measures of complexity of tensor networks and studied their properties. In this work (part II), we show how classical approximation tools, such as polynomials or splines (with fixed or free knots), can be encoded as a tensor network with controlled complexity. We use this to derive direct (Jackson) inequalities for the approximation spaces of tensor networks. This is then utilized to show that Besov spaces are continuously embedded into these approximation spaces. In other words, we show that arbitrary Besov functions can be approximated with optimal or near to optimal rate. We also show that an arbitrary function in the approximation class possesses no Besov smoothness, unless one limits the depth of the tensor network.

arXiv commentFor part I see arXiv:2007.00118, for part III see arXiv:2101.11932

References in corpus (16)

• Deep Learning in Neural Networks: An Overview
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States
• An Area Law for One Dimensional Quantum Systems
• Tensor networks for complex quantum systems
• Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1
• Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
• Quantum Entanglement in Deep Learning Architectures
• Adaptive Near-Optimal Rank Tensor Approximation for High-Dimensional Operator Equations
• On the Dirac-Frenkel Variational Principle on Tensor Banach Spaces
• Approximating local observables on projected entangled pair states
• Learning with tree tensor networks: complexity estimates and model selection
• Approximation of Smoothness Classes by Deep Rectifier Networks
• Higher-order principal component analysis for the approximation of tensors in tree-based low-rank formats
• Hackbusch Conjecture on tensor formats
• Tensor-based multiscale method for diffusion problems in quasi-periodic heterogeneous media

Cited by in corpus (2)

• Learning with tree tensor networks: complexity estimates and model selection
• A Quantum-Inspired Algorithm for Wave Simulation Using Tensor Networks