Seq2Tens: An Efficient Representation of Sequences by Low-Rank Tensor Projections
arXiv:2006.07027
Abstract
Sequential data such as time series, video, or text can be challenging to analyse as the ordered structure gives rise to complex dependencies. At the heart of this is non-commutativity, in the sense that reordering the elements of a sequence can completely change its meaning. We use a classical mathematical object -- the tensor algebra -- to capture such dependencies. To address the innate computational complexity of high degree tensors, we use compositions of low-rank tensor projections. This yields modular and scalable building blocks for neural networks that give state-of-the-art performance on standard benchmarks such as multivariate time series classification and generative models for video.
37 pages, 6 figures, 8 tables
References in corpus (10)
- Sequence to Sequence Learning with Neural Networks
- WaveNet: A Generative Model for Raw Audio
- Improving Generalization Performance by Switching from Adam to SGD
- Should we really use post-hoc tests based on mean-ranks?
- Rough paths, Signatures and the modelling of functions on streams
- Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU
- Autoregressive Kernels For Time Series
- Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited
- Cumulants, free cumulants and half-shuffles
- A Generalised Signature Method for Multivariate Time Series Feature Extraction